Your tasks
- Design, develop, and operate scalable and maintainable data pipelines in the Azure Databricks environment
- Develop all technical artefacts as code, implemented in qualified IDEs, with full version control and CI/CD automation
- Enable data-driven decision-making in Supply Chain Management (SCM) by ensuring high data availability, quality, and reliability
- Implement data products and analytical assets using software engineering principles in close alignment with business domains and functional IT
- Apply rigorous software engineering practices such as modular design, test-driven development, and artifact reuse in all implementations
- Global delivery footprint; cross-functional data engineering support across SCM domains
- Collaboration with business stakeholders, functional IT partners, product owners, architects, ML/AI engineers, and Power BI developers
- Agile, product-team structure embedded in an enterprise-scale Azure environment

Main Tasks:
- Design scalable batch and streaming pipelines in Azure Databricks using PySpark and/or Scala




- Implement ingestion from structured and semi-structured sources (e.g., SAP, APIs, flat files)
- Build bronze/silver/gold data layers following the defined lakehouse layering architecture & governance
- Implement use-case driven dimensional models (star/snowflake schema) tailored to SCM needs
- Ensure compatibility with reporting tools (e.g., Power BI) via curated data marts and semantic models
- Implement enterprise-level data warehouse models (domain-driven 3NF models) for SCM data, closely aligned with data engineers for other business domains
- Develop and apply master data management strategies (e.g., Slowly Changing Dimensions)
- Develop automated data validation tests using frameworks
- Monitor pipeline health, identify anomalies, and implement quality thresholds
- Establish data quality transparency by defining and implementing meaningful data quality rules with source system and business stakeholders a

📌 IT engineer data lakehouse (Bengaluru)
🏢 Continental
📍 Bengaluru