Exp: 6 to 11 yrs
Key Responsibilities
Design and develop scalable data pipelines using Databricks (Spark, PySpark, SQL)
Build and optimize ETL/ELT workflows for large-scale data processing
Work with Delta Lake for data management, performance tuning, and reliability
Collaborate with data scientists to productionize ML models on Databricks
Implement ML pipelines including data preparation, feature engineering, and model deployment
Ensure data quality, governance, and security best practices
Integrate Databricks solutions with cloud platforms (AWS/Azure/GCP)
Monitor, troubleshoot, and optimize performance of data workflows
Contribute to CI/CD pipelines and automation for data and ML workflows

Required Skills & Qualifications

- 6–11 years of experience in Data Engineering / Big Data ecosystem
- Strong hands-on experience with Databricks, Apache Spark, and PySpark
- Proficiency in SQL and Python
- Experience with Delta Lake, Unity Catalog (preferred)




- Strong understanding of data modeling and data warehousing concepts
- Experience working on cloud platforms (AWS / Azure / GCP)
- Exposure to Machine Learning workflows (training, deployment, monitoring)
- Familiarity with ML libraries like scikit-learn, MLflow, TensorFlow, or similar
- Experience with workflow orchestration tools (Airflow, ADF, etc.)
- Strong problem-solving and debugging skills

Good to Have
Experience with MLflow for experiment tracking and model management
Knowledge of MLOps practices
Experience in real-time/streaming pipelines (Kafka, Structured Streaming)
Exposure to Data Governance & Security frameworks
Certification in Databricks or cloud platforms

Soft Skills
Solid communication and stakeholder management skills
Ability to work in agile, fast-paced environments
Collaborative mindset with a focus on continuous learning

📌 Software/Scrum team lead (Bengaluru)
🏢 Digital Impetus
📍 Bengaluru