Overview

Senior Data Engineer - Databricks & Modern Data Platforms :

- Role Summary**

Databricks & Lakehouse Data Engineering:

- Design, build, and optimize high-performance batch and streaming pipelines using Databricks (Apache Spark, PySpark, Delta Lake).
- Implement medallion architecture (Bronze, Silver, Gold layers) with clear data contracts.
- Ensure reliability, scalability, performance optimization, and cost-effective processing.
- Leverage Delta Lake capabilities such as ACID transactions, schema evolution, and time travel.

Advanced Data Modeling & Analytics Enablement

- Design curated, analytics-ready datasets including dimensional and fact-based models.
- Build reusable transformations and semantic layers for BI and analytics.
- Make informed decisions on table materialization vs. views.

Customer & Event Data Engineering

- Engineer scalable customer, identity, household, and event-based data models.
- Support Customer 360-style datasets with consistent metric definitions.
- Ensure alignment between operational data models and reporting consumption.

Legacy Data Modernization & Migration

- Reverse engineer legacy reporting logic and migrate workloads to Databricks.
- Support data reconciliation, validation, and parallel run strategies.
- Document and communicate metric changes during modernization.

Data Engineering Best Practices





- Treat data pipelines and SQL as production-grade code with Git-based version control.
- Implement CI/CD pipelines, peer reviews, data quality checks, and monitoring.
- Maintain explicit documentation, lineage awareness, and ownership definitions.

Collaboration & Stakeholder Engagement

- Collaborate with Analytics, BI, Product, Marketing, and Data Science teams.
- Translate business requirements into scalable technical solutions.
- Clearly explain data logic and platform changes to non-technical stakeholders.

Required Qualifications

- 6-8 years of experience in Data Engineering.
- 4-6 years of hands-on experience with Databricks, Apache Spark, PySpark, and Delta Lake.
- Strong SQL expertise on distributed data platforms.
- Solid understanding of data modeling, lakehouse architecture, and analytics enablement.
- Experience with orchestration tools such as Azure Data Factory, Airflow, or similar.
- Experience working with cloud platforms (Azure, AWS, or GCP).

Good to Have - AI & Modern Data Tooling

- Working knowledge of AI/ML fundamentals and data engineering support for ML workloads.
- Exposure to Databricks MLflow, Feature Store, or Model Serving.
- Familiarity with GenAI / LLM ecosystems such as OpenAI, AWS Bedrock, LangChain, Langgraph
- Basic understanding of prompt engineering, embeddings, or AI-assisted analytics.

Pay Range

INR ₹0.00 - INR ₹0.00 /Yr.

📌 Senior Engineer 1 (Bengaluru)
🏢 JCPenney
📍 Bengaluru