Senior Data Engineer

Job Summary

Position Purpose: The Data Analytics organization enables Bayer s Corporate Functions-including Finance, Supply Chain Logistics, HR, Procurement, Legal, and Sustainability-to become more insight and data-driven. Our mission is to deliver a robust data foundation and FAIR-Findable, Accessible, Interoperable, and Reusable-data assets that improve business decision-making.

Responsibilities

- Design, build, and maintain scalable, reliable, and cost-efficient data pipeline architectures.
- Develop idempotent, automated ETL/ELT pipelines using Python, PySpark, Delta Lake, and MERGE INTO patterns.
- Drive automation across deployments, monitoring, incident remediation, and operational processes to improve efficiency, quality, and reliability.
- Implement robust DataOps practices, including data-contract enforcement, automated data-quality checks, and unit and integration testing.
- Establish fine-grained data governance, lineage tracking, and security access controls across environments using Unity Catalog.
- Define and manage Infrastructure as Code using Terraform and Azure DevOps for Databricks workspaces, compute clusters, serverless SQL warehouses, and storage integrations.
- Build and maintain automated CI/CD pipelines using Databricks Asset Bundles, Databricks CLI, GitHub Actions,



Azure DevOps, or GitLab.
- Monitor compute consumption and costs, optimize cluster configurations, and implement cluster policies to promote effective resource utilization.

Requirements

- At least five years of experience in data engineering, DataOps, DevOps, or a comparable technical role.
- Advanced hands-on experience with Python, PySpark, SQL, Git, and Databricks.
- Strong experience with the Databricks ecosystem, including Delta Lake, Unity Catalog, and Databricks Serverless SQL.
- Proven experience with Infrastructure as Code, preferably Terraform, and CI/CD tooling such as Databricks Asset Bundles, GitHub Actions, Azure DevOps, or GitLab.
- Experience implementing data-quality controls, data contracts, automated testing, monitoring, and incident-management practices.
- Strong knowledge of Azure cloud services, including security, networking, identity and access management, and storage integration. Experience with AWS or GCP is beneficial.

Location

Bangalore

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Sr Data Engineer (Bengaluru)
🏢 Bayer Zydus Pharma
📍 Bengaluru