Job Title: ETL Developer
Position: Software Engineer
Experience: 4- 7 Years
Category: Software Development/ Engineering
Shift: 1-10PM (Hybrid)
Main location: Bangalore / Chennai
Position ID: J0726-1184
Employment Type: Full Time
Education Qualification: Bachelor's degree in Computer Science or related field or higher with minimum 3 years of relevant experience.
LI-MP14

Your future duties and responsibilities
- Design, develop, test, deploy, and support ETL and ELT pipelines using Azure Databricks.
- Build scalable data transformation workflows using PySpark, Spark SQL, and Databricks notebooks.
- Develop reusable Databricks frameworks, utilities, and common pipeline components.
- Design and implement batch and near-real-time data pipelines.
- Build and manage Databricks jobs, workflows, schedules, dependencies, retries, and alerts.
- Develop data cleansing, validation, enrichment, reconciliation, and transformation logic.
- Implement data quality controls, audit checks, exception handling, and restart capabilities.
- Optimize Spark workloads, SQL queries,



cluster configuration, partitioning, caching, and file formats.
- Work with Delta Lake to support reliable, scalable, and transactional data processing.
- Design and maintain bronze, silver, and gold data layers within a lakehouse architecture.
- Support source-to-target mapping, data profiling, data lineage, and technical design activities.
- Develop and maintain SQL queries, stored procedures, views, and data reconciliation scripts.
- Integrate Databricks pipelines with Azure Data Lake Storage and other Azure services.
- Build and maintain automated deployment processes using Azure DevOps CI/CD pipelines.
- Promote code, notebooks, jobs, and configuration across development, testing, and production environments.
- Monitor pipeline performance and troubleshoot failures, data issues, and production incidents.
- Perform root-cause analysis and implement permanent fixes for recurring issues.
- Create and maintain te

📌 Senior Software Engineer- Etl Developer (Chennai)
🏢 CGI
📍 Chennai