Responsible for designing, developing, and maintaining data pipelines and ETL processes using Databricks and Azure Data Factory.

- Valuable experience in using SQL, Python and Spark within Databricks notebooks for data processing and transformations.

- Integrate and orchestrate data from various sources, ensuring data quality and accuracy.

- Build reusable rules for Data Quality and required transformations.

- Optimize, monitor data pipelines for performance and scalability and debug in case of issues.

- Proficiency in Databricks, Azure Data Factory, Python, SQL is must.

- Familiarity with ETL principles, data warehousing, and big data technologies.

- Evaluate and define technical specs using BRD for BI and DW solutions.

- Build conceptual and logical models based on the functional flow of business in scalable way.

- Work with Business functions/ leadership and Application SMEs to understand the requirement and analysing the source to fulfil the requirements.

- Be compliant to data quality, governance, compliance, and other legal requirements.

- Act as key P.O.C between data analyst, data scientists,



and the business/application teams.

- Must have Azure Databricks experience & Data Integrations.

- Good work experience on Data bricks, ADF, and other Azure and Databricks objects.

- Experience on creating frameworks towards building the optimized data pipelines is a must.

- Working knowledge of different self-hosted and other runtimes along with SSIS.

- Strong work experience of SQL, Scala, Python and Spark.

- Performance Tuning of Spark SQL running on DELTA LAKE storage and Strong Knowledge on Databricks and Cluster Configurations.

- Working knowledge of Data Warehousing, Modelling, Data Validation, Data Migration.

- Documentation of all project resources and follow best practices for development.

- Knowledge of front-end tools like Qlik Sense, MS PBI

- Minimum bachelors degree in computer science or related field is required.

Strong problem-solving skills, good communication abilities and go-getter attitude.

📌 Data Engineer-Information Technology (Pune)
🏢 Emcure
📍 Pune