Senior Data Engineer - Databricks / Fabric & Azure Data Factory (Coimbatore)

Senior Data Engineer - Databricks / Fabric & Azure Data Factory (Coimbatore)

17 Nov
|
Chosen Talentz
|
Coimbatore

17 Nov

Chosen Talentz

Coimbatore

Job Summary

We are looking for an experienced Data Engineer with strong hands-on expertise in Databricks or Microsoft Fabric and Azure Data Factory (ADF) to design, develop, and maintain scalable data engineering solutions.

The ideal candidate should have experience in building data pipelines, implementing ETL/ELT processes, transforming and integrating data from multiple sources, and ensuring data quality and reliability. Experience with SQL, Python, data modelling, and cloud-based data platforms is essential.

Position Details

- Employment Type: Full time
- Work Mode: Work from Office
- Location: Coimbatore, India

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and either Databricks or Microsoft Fabric.
- Build and maintain data ingestion pipelines using Azure Data Factory (ADF), integrating data from approved files, APIs, and other source systems. Manage incremental loads, schema changes, failed loads, error handling, and recovery.
- Develop and maintain Bronze, Silver, and Gold data layers using Azure Databricks or Microsoft Fabric, implementing scalable data transformation logic. Experience with dbt for developing modular data models, reusable transformation logic, testing, and documenting dependencies is an added advantage.
- Implement source-to-target mappings, reconciliation, incremental processing, historical data handling, and dimensional models.
- Ensure pipeline data quality through validation rules, completeness and freshness checks, exception handling, reconciliation, and remediation tracking. Implement automated data quality tests using dbt or platform-native capabilities.
- Follow engineering standards and best practices in pipeline development, participate in code and design reviews, and coordinate technical or architectural decisions with the onshore architect.
- Coordinate data platform and Profisee data flows with the MDM engineer and trusted reporting datasets with the BI engineer.
- Maintain technical metadata,



transformation documentation, and lineage in coordination with the MDM and Data Governance Engineer.
- Coordinate integration testing, defect resolution, deployment readiness, and production handover; provide evidence for release approval.
- Identify delivery risks early and coordinate priorities with the Project Manager, CRM team, and client stakeholders.

Required Skills

- 5+ years in data engineering and ETL development.
- Strong hands-on SQL, Databricks or Microsoft Fabric.
- Mandatory hands-on experience with Azure Data Factory (ADF) for data ingestion, orchestration, and pipeline development.
- Experience building and maintaining ETL/ELT pipelines and scalable data integration solutions.
- Strong understanding of Data Lakehouse architecture, Medallion architecture (Bronze, Silver, Gold), and data warehousing concepts.
- Experience with source-to-target mapping, data reconciliation, incremental loading, historical data handling, and dimensional modeling.
- Experience delivering at least two production data platform implementations, including source reconciliation and operational handover.
- Working knowledge of MDM integration patterns, data governance controls, and Power BI data requirements.

Good to Have

- Experience with Fivetran or other data ingestion / ELT tools.
- Experience with dbt models, tests, documentation, incremental strategies, Git workflows, and performance tuning.
- Financial services or investment data experience, including customer, account, advisor, or sales data.
- Exposure to Profisee, Dynamics 365 integrations, Python, and Azure DevOps.
- Databricks/Microsoft Fabric or Azure certifications.
- Consulting or systems integrator delivery experience.

Additional Requirements

- Bachelors degree in Computer Science, IT, Engineering, or a related field, or equivalent relevant experience.
- Solid written and verbal communication, structured documentation and ability to take ownership of assigned tasks.
- Availability for agreed overlap with US Central business hours and planned releases is expected.

📌 Senior Data Engineer - Databricks / Fabric & Azure Data Factory (Coimbatore)
🏢 Chosen Talentz
📍 Coimbatore

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: senior data engineer - databricks / fabric & azure data factory (coimbatore) / coimbatore

Subscribe to this job alert:

Get the latest job offers by email for: senior data engineer - databricks / fabric & azure data factory (coimbatore) / coimbatore