Position Description
Bangalore/Chennai Employment Type: Full Time
Job Summary:
We are seeking a highly experienced Senior Azure Data Engineer to join our growing data team. In this role, you will be responsible for designing, developing, and operationalizing large scale data processing systems on the Azure cloud platform. The ideal candidate will have a deep understanding of modern data architecture, a passion for building productive and reliable data pipelines, and extensive hands on experience with Azure Databricks, Azure Data Lake, PySpark, and Python.
Work Mode:
Full Time / 5 days Work From Office/ ODC (Banking client)
Your future duties and responsibilities
Key Responsibilities:
• Design, build, and maintain scalable and robust data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
• Develop and optimize high performance Spark jobs using PySpark and Spark SQL within Azure Databricks.
• Implement data storage solutions using Azure Data Lake Storage (Gen2) following medallion architecture (Bronze, Silver, Gold layers)
and best practices for data organization.
• Collaborate with data architects, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
• Implement data security and compliance measures, including access controls, encryption, and data masking within the Azure ecosystem.
• Perform data modeling to create efficient, scalable schemas for both batch and real time analytics.
• Monitor, troubleshoot, and tune data pipelines and Databricks jobs for performance and cost effectiveness.
• Automate deployment and management of data solutions using CI/CD pipelines (Azure DevOps/GitHub Actions) and Infrastructure as Code (IaC) tools like Terraform or Bicep.
• Establish and enforce data quality checks and data governance standards across the data platform.
• Mentor junior data engineers and promote best practices in software development and data en
📌 Azure Data Engineer (Bengaluru)
🏢 CGI
📍 Bengaluru