Job Title: Azure Data Engineer
Position: Software Engineer
Experience: 5- 7 Years
Category: Software Development/ Engineering
Shift: 1-10PM (Hybrid)
Main location: Chennai
Position ID: J0726-1184
Employment Type: Full Time
Education Qualification: Bachelor's degree in Computer Science or related field or higher with minimum 3 years of relevant experience.

Position Description: We are looking for an experienced Azure Data Developer to join our team. The ideal candidate should be passionate about coding and developing scalable and high-performance applications. The successful candidate will be responsible for designing, developing, optimizing, and supporting enterprise-grade ETL and ELT pipelines primarily using Azure Databricks, PySpark, SQL, and related Microsoft Azure services. This is a hands-on engineering role focused on building scalable, reliable, and high-performing data pipelines across development, testing, and production environments. The engineer will work closely with data architects, business analysts, application teams, quality assurance teams, and production support teams to deliver secure and well-governed data solutions in a regulated financial-services workplace.

Your future duties and responsibilities
• Design, develop, test, deploy, and support ETL and ELT pipelines using Azure Databricks.
• Build scalable data transformation workflows using PySpark, Spark SQL, and Databricks notebooks.
• Develop reusable Databricks frameworks, utilities, and common pipeline components.
• Ingest and process data from relational databases, flat files, APIs, cloud storage,



and enterprise applications.
• Design and implement batch and near-real-time data pipelines.
• Build and manage Databricks jobs, workflows, schedules, dependencies, retries, and alerts.
• Develop data cleansing, validation, enrichment, reconciliation, and transformation logic.
• Implement data quality controls, audit checks, exception handling, and restart capabilities.
• Optimize Spark workloads, SQL queries, cluster configuration, partitioning, caching, and file formats.
• Work with Delta Lake to support reliable, scalable, and transactional data processing.
• Design and maintain bronze, silver, and gold data layers within a lakehouse architecture.
• Support source-to-target mapping, data profiling, data lineage, and technical design activities.
• Develop and maintain SQL queries, stored procedures, views, and data reconciliation scripts.
• Integrate Databricks pipelines with Azure Data Lake Storage and other Azure services.
• Build and maintain automated deployment processes using Azure DevOps CI/CD pipelines.
• Promote code, notebooks, jobs, and configuration across development, testing, and production environments.
• Monitor pipeline performance and troubleshoot failures, data issues, and production incidents.
• Perform root-cause analysis and implement permanent fixes for recurring issues.
• Create and maintain technical documentation, data mappings, deployment procedures, and support runbooks.
• Ensure solutions comply with CIBC security, data governance, risk management, and regulatory requirements.

📌 Senior Software Engineer- Azure Data Engineer (Tamil Nadu)
🏢 CGI
📍 Tamil Nadu