Role & Responsibilities

- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Databricks, PySpark, Python, and SQL.

- Build and optimize data processing workflows using Apache Spark and Delta Lake.

- Implement data ingestion and transformation pipelines using Azure Data Factory (ADF) and ADLS.

- Work on batch and incremental data processing, data integration, and data warehousing solutions.

- Optimize Spark jobs and SQL queries for better performance and cost efficiency.

- Implement data quality checks, schema validation, and source-to-target reconciliation.

- Monitor, troubleshoot, and resolve pipeline failures and performance issues.

- Collaborate with cross-functional teams to understand business requirements and deliver scalable data solutions.

Preferred Candidate Profile





- 5+ years of experience in Data Engineering with strong hands-on expertise in Azure Databricks.

- Strong proficiency in PySpark, Apache Spark, Python, and SQL.

- Hands-on experience with Delta Lake, Azure Data Factory, and ADLS Gen2.

- Experience in building end-to-end ETL/ELT pipelines and handling batch and incremental data processing.

- Knowledge of Spark performance tuning, query optimization, and data quality frameworks.

- Exposure to Azure DevOps, GitHub, CI/CD pipelines, and deployment processes.

- Robust problem-solving, analytical, and communication skills.

📌 Hiring For Azure Data Engineer - Pan India (Bengaluru)
🏢 Crescendo Global Leadership Hiring India
📍 Bengaluru