Job DescriptionExp: 6 to 11 yrsKey Responsibilities- Design and develop scalable data pipelines using Databricks (Spark, PySpark, SQL)- Build and optimize ETL/ELT workflows for large-scale data processing- Work with Delta Lake for data management, performance tuning, and reliability- Collaborate with data scientists to productionize ML models on Databricks- Implement ML pipelines including data preparation, feature engineering, and model deployment- Ensure data quality, governance, and security best practices- Integrate Databricks solutions with cloud platforms (AWS/Azure/GCP)- Monitor, troubleshoot, and optimize performance of data workflows- Contribute to CI/CD pipelines and automation for data and ML workflowsRequired Skills & Qualifications- 6–11 years of experience in Data Engineering / Big Data ecosystem- Strong hands-on experience with Databricks, Apache Spark, and PySpark- Proficiency in SQL and Python- Experience with Delta Lake,



Unity Catalog (preferred)- Strong understanding of data modeling and data warehousing concepts- Experience working on cloud platforms (AWS / Azure / GCP)- Exposure to Machine Learning workflows (training, deployment, monitoring)- Familiarity with ML libraries like scikit-learn, MLflow, TensorFlow, or similar- Experience with workflow orchestration tools (Airflow, ADF, etc.)- Strong problem-solving and debugging skillsGood to Have- Experience with MLflow for experiment tracking and model management- Knowledge of MLOps practices- Experience in real-time/streaming pipelines (Kafka, Structured Streaming)- Exposure to Data Governance & Security frameworks- Certification in Databricks or cloud platformsSoft Skills- Strong communication and stakeholder management skills- Ability to work in agile, quick-paced environments- Collaborative mindset with a focus on continuous learning

📌 Lead Software Engineer (Bengaluru)
🏢 Impetus Capital
📍 Bengaluru