Azure Synapse Analytics, Azure Data Lake Storage (ADLS), PySpark, Delta Lake, CI/CD for data pipelines, AZURE DATAFACTORY

Key Responsibilities

- Design, develop, and maintain end-to-end data pipelines using Azure Data Factory for batch and scheduled workloads.
- Build and orchestrate data transformations and processing workflows using Databricks aligned to business requirements.
- Implement robust pipeline monitoring, alerting, and failure handling to ensure reliable and repeatable executions.
- Perform data validation and reconciliation checks to ensure accuracy, completeness, and consistency across sources and targets.
- Optimize pipeline performance by tuning ADF activities, improving orchestration logic, and streamlining transformations in Databricks.
- Collaborate with cross-functional teams to gather requirements, estimate effort, and deliver solutions within timelines.
- Maintain technical documentation for pipelines, datasets, schedules, dependencies, and operational runbooks.
- Support deployments and workplace promotions by following structured release and change management practices.

Minimum

Qualifications:

- Education: BTECH, MTECH, MCA, MSC.




- Overall experience: 3–5 years in data engineering / data integration roles.
- Hands-on experience with Azure Data Factory including pipeline development, scheduling, triggers, and integration patterns.
- Hands-on experience with Databricks for data processing and transformation workflows.
- Ability to troubleshoot pipeline failures, analyze logs, and implement corrective actions to improve stability.
- Experience designing scalable orchestration patterns in ADF (parameterization, reusable components, dependency handling).
- Strong understanding of data transformation best practices and implementing efficient processing in Databricks.
- Exposure to building data quality checks and operational dashboards for pipeline health and SLA tracking.
- Experience collaborating with stakeholders to translate business requirements into technical pipeline designs and delivery plans.
- Familiarity with performance tuning approaches for cloud data pipelines and distributed processing workloads.

📌 Azure Datafactory (Bengaluru)
🏢 Infosys
📍 Bengaluru