Key Responsibilities:

- Design, develop, and maintain scalable data pipelines using Azure Databricks and Azure Data Factory.
- Build robust ETL/ELT workflows for batch and real-time data processing.
- Develop data transformations using PySpark, Python, and SQL.
- Design and implement data lake/lakehouse and data warehouse solutions.
- Integrate data from multiple structured and unstructured sources.
- Optimize data pipelines, queries, and Spark workloads for performance and cost.
- Implement data quality, validation, monitoring, and error-handling mechanisms.
- Work with cloud-based data storage and Azure data services.
- Collaborate with architects, data scientists, analysts, and application teams.
- Follow engineering best practices around CI/CD, security, governance, and documentation.





Required Skills :

- 8 to 15 years of experience in data engineering.
- Strong hands-on experience with Azure Databricks.
- Strong hands-on experience with Azure Data Factory (ADF).
- Excellent PySpark, Python, and SQL skills.
- Strong understanding of ETL/ELT concepts and data pipeline architecture.
- Experience with Data Lake / Lakehouse architecture.
- Experience with data modeling and performance optimization.
- Positive understanding of Microsoft Azure data services.
- Experience working in enterprise-scale data environments.

📌 Data Engineer - ETL/Python (India)
🏢 Andor Tech
📍 India