Role & Responsibilities
Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Databricks, PySpark, Python, and SQL.
Build and optimize data processing workflows using Apache Spark and Delta Lake.
Implement data ingestion and transformation pipelines using Azure Data Factory (ADF) and ADLS.
Work on batch and incremental data processing, data integration, and data warehousing solutions.
Optimize Spark jobs and SQL queries for better performance and cost efficiency.
Implement data quality checks, schema validation, and source-to-target reconciliation.
Monitor, troubleshoot, and resolve pipeline failures and performance issues.
Collaborate with cross-functional teams to understand business requirements and deliver scalable data solutions.

Preferred Candidate Profile




5+ years of experience in Data Engineering with robust hands-on expertise in Azure Databricks.
Solid proficiency in PySpark, Apache Spark, Python, and SQL.
Hands-on experience with Delta Lake, Azure Data Factory, and ADLS Gen2.
Experience in building end-to-end ETL/ELT pipelines and handling batch and incremental data processing.
Knowledge of Spark performance tuning, query optimization, and data quality frameworks.
Exposure to Azure DevOps, GitHub, CI/CD pipelines, and deployment processes.
Robust problem-solving, analytical, and communication skills.

📌 Hiring For Azure Data Engineer Pan India Bengaluru
🏢 Crescendo Global Leadership Hiring India
📍 India