Posted on: 13-10-2026

Experience: 8-12 years

India-Bengaluru

Job ID #1002281

Role Requirements

- Skills: Python, OOP, Multiprocessing, Asyncio, Decorators, Generators, Pytest, ETL/ELT, Databricks, Delta Lake, Structured Streaming, Auto Loader, Unity Catalog, Apache Spark, T-SQL, SQL, Dimensional Data Modeling, Star Schema, Snowflake Schema, SCD Types 1/2/3, Data Vault, Kimball, Inmon, Databricks Workflows, Airflow, Azure Data Factory, Azure, AWS, CI/CD
- Design, build, and own production-grade Python codebases for the data platform, including ETL/ELT pipelines, reusable internal packages, schema validation, and testing frameworks.
- Write clean, well-tested, performant Python using OOP, concurrency, decorators, and generators to solve pipeline-scale problems.
- Profile and optimize slow Python pipelines through algorithmic and code-level improvements, migrating heavy workloads to Databricks/Spark when Python alone will not scale.
- Set and enforce Python engineering standards across the team, including code reviews, packaging/versioning, structured logging, and exception-handling patterns.Design and deliver Bronze/Silver/Gold medallion pipelines on Databricks using Delta Lake, Structured Streaming/Auto Loader, and Unity Catalog governance.
- Own dimensional data modeling decisions, including star/snowflake schema design and SCD strategy, for core platform data models.
- Write and review advanced T-SQL for platform workloads, including window functions, MERGE/upsert logic, and execution-plan-based performance tuning.
- Partner with Data Architects and stakeholders on ETL versus ELT, orchestration, and cost/performance trade-offs across the platform.
- Mentor Data Engineers and Senior Data Engineers through design and code reviews.
- Demonstrate 8–12 years of data engineering experience,



including at least 2 years in technical leadership or mentoring.
- Have strong hands-on Python experience, including OOP, generators, decorators, multiprocessing/asyncio, exception handling, and unit testing with pytest.
- Have experience building production ETL/ELT pipelines, schema validation, and reusable internal Python packages/frameworks.
- Demonstrate the ability to profile, debug, and optimize Python code for data-pipeline performance.
- Have deep, hands-on Databricks experience with Delta Lake, Structured Streaming, Auto Loader, and Unity Catalog.
- Experience tuning Spark jobs at scale, including data skew, cluster sizing/autoscaling, and cost optimization.
- Demonstrate robust dimensional modeling knowledge, including fact/dimension grain and conformed dimensions, with working knowledge of Data Vault and Kimball versus Inmon trade-offs.
- Have advanced T-SQL skills, including window functions, MERGE/upsert patterns, indexing strategy, and execution-plan-based query tuning.
- Have experience designing medallion architectures and evaluating ETL versus ELT and batch versus streaming trade-offs.
- Have experience with orchestration using Databricks Workflows, Airflow, or ADF on Azure or AWS.
- PCEP/PCAP or equivalent Python certification and Databricks Certified Data Engineer Professional certification are good to have.
- Experience with data governance/compliance, GDPR/HIPAA, FinOps, and formally leading or mentoring a team of 3+ engineers is preferred.
- A Bachelor’s or Master’s degree in Computer Science, Engineering, a related field, or equivalent practical experience.
- Demonstrate strong stakeholder communication and the ability to explain technical trade-offs to engineering and business audiences while owning ambiguous, enterprise-scale design decisions.

📌 Data Engineer - ATA (Bengaluru)
🏢 Kalpita Technologies
📍 Bengaluru