Data Engineer –

We are looking for a Data Engineer to design, build, and maintain scalable data pipelines that support analytics and data products. The role involves working closely with data scientists and engineering teams to ensure reliable and efficient data infrastructure.

Key Responsibilities

Design and develop data pipelines (batch and streaming) using Databricks and PySpark.

Write optimized PySpark code to process large datasets.

Build and manage CI/CD pipelines for data workflows using GitHub / GitHub Actions / Azure DevOps.

Implement data validation and data quality checks.

Monitor pipelines using logging, alerting, and observability tools.

Collaborate with cross-functional teams and document data pipeline architecture.





Mandatory Skills

7 years of Data Engineering experience

Databricks

PySpark

End-to-end Data Pipeline Development (Batch / Streaming)

GitHub (branching, pull requests, code reviews)

CI/CD for Data Pipelines (GitHub Actions / Azure DevOps)

Data Validation / Data Quality Frameworks

Pipeline Monitoring (Logging, Alerting, SLA tracking)

Apache Kafka

Data Warehousing / Lakehouse Architecture concepts

Positive to Have

MongoDB or other NoSQL databases

Data Governance

Data Modeling

Spark Streaming

📌 Data Engineer (India)
🏢 Sparix Global
📍 India