Senior Data Engineer

POSITION PURPOSE:
• The Data & Analytics organization enables Bayer’s Corporate Functions—including Finance, Supply Chain & Logistics, HR, Procurement, Legal, and Sustainability—to become more insight and data-driven. Our mission is to deliver a robust data foundation and FAIR—Findable, Accessible, Interoperable, and Reusable—data assets that improve business decision-making.

YOUR TASKS AND RESPONSIBILITIES:
• Design, build, and maintain scalable, reliable, and cost-efficient data pipeline architectures.
• Develop idempotent, automated ETL/ELT pipelines using Python, PySpark, Delta Lake, and MERGE INTO patterns.
• Drive automation across deployments, monitoring, incident remediation, and operational processes to improve efficiency, quality, and reliability.
• Implement robust DataOps practices, including data-contract enforcement, automated data-quality checks, and unit and integration testing.
• Establish fine-grained data governance, lineage tracking, and security access controls across environments using Unity Catalog.




• Define and manage Infrastructure as Code using Terraform and Azure DevOps for Databricks workspaces, compute clusters, serverless SQL warehouses, and storage integrations.
• Build and maintain automated CI/CD pipelines using Databricks Asset Bundles, Databricks CLI, GitHub Actions, Azure DevOps, or GitLab.
• Monitor compute consumption and costs, optimize cluster configurations, and implement cluster policies to promote productive resource utilization.

WHO YOU ARE:
• At least five years of experience in data engineering, DataOps, DevOps, or a comparable technical role.
• Advanced hands-on experience with Python, PySpark, SQL, Git, and Databricks.
• Strong experience with the Databricks ecosystem, including Delta Lake, Unity Catalog, and Databricks Serverless SQL.
• Proven experience with Infrastructure as Code, preferably Terraform, and CI/CD tooling such as Databricks Asset Bundles, GitHub Actions, Azure DevOps, or GitLab.

📌 Senior Data Engineer (Bengaluru)
🏢 Bayer
📍 Bengaluru