12 Nov
|
EXL
|
Maharashtra
Role: Databricks MLOps Engineer
Experience: 6+ Years of IT Experience (3+ years building ML pipelines on Databricks)
We are seeking a Databricks MLOps Engineer with 6+ years in ML engineering, MLOps, data engineering, including 3+ years building ML pipelines on Databricks, to take models from notebooks to tested, monitored production. The role covers feature pipelines, training automation, model registry, serving, monitoring and CI/CD for ML, and supports LLM and GenAI workloads with clear cost controls.
- Build feature pipelines on Delta tables and manage features in Unity Catalog feature tables.
- Automate training and retraining with Databricks Workflows, including parameterized and scheduled runs.
- Track experiments and register, version and promote models with MLflow and the Unity Catalog model registry, using aliases for champion and challenger models.
- Deploy models to Databricks Model Serving (real-time) and batch inference jobs; run A/B and shadow deployments.
- Monitor models with Lakehouse Monitoring and inference tables: data drift, prediction drift, latency, error rates and alert thresholds.
- Implement CI/CD for ML with Databricks Asset Bundles and GitHub Actions or Azure DevOps: tests, staged promotion and rollback.
- Support LLM and GenAI workloads: serving endpoints, AI Gateway rate limits, model-tier selection and token usage tracking per endpoint.
- Partner with data scientists to turn notebooks into tested, packaged production code.
Required Technical Skills Core Technologies
- Databricks Lakehouse Platform
- MLflow (tracking, registry, deployment)
- Python for production
- PySpark and Delta Lake
ML Engineering
- Feature engineering and Unity Catalog feature tables
- Training automation with Databricks Workflows
- Model Serving and batch inference
- Champion/challenger promotion, A/B and shadow testing
Monitoring & Governance
- Lakehouse Monitoring and inference tables
- Data and prediction drift detection, alerting and retraining triggers
- Unity Catalog model governance and access control
DevOps
- Databricks Asset Bundles
- CI/CD with GitHub Actions or Azure DevOps
- Unit and integration testing (pytest)
- Git-based workflows
LLM / GenAI Operations
- Serving endpoints and AI Gateway
- Token usage tracking, model-tier selection and cost per call
- Vector Search for RAG use cases
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Data Science, Engineering or a related field.
- 6+ years of experience in ML engineering, MLOps or data engineering with ML delivery.
- 3+ years of hands-on experience building ML pipelines on Databricks (Workflows, MLflow, Model Serving or batch inference).
- Multiple models deployed to production with monitoring, retraining and rollback in place.
- Robust Python engineering practices: packaging, testing, logging and error handling.
Preferred Qualifications
- Databricks Certified Machine Learning Associate or Professional.
- Experience with Feature Store and online tables.
- Experience supporting LLM or GenAI applications in production.
- Docker, REST APIs and basic cloud networking for serving endpoints.
Soft Skills
- Ability to explain model behaviour, drift and risk to business stakeholders in plain language.
- Strong collaboration with data scientists, data engineers and product teams.
- Experience working in Agile/Scrum environments.
- Clear documentation of pipelines, runbooks and release processes.
Nice to Have
- Azure Machine Learning or Amazon SageMaker
- Kubernetes
- Media, telecom or sports analytics use cases (churn, recommendations, audience)
📌 Databricks MLOps Engineer (Maharashtra)
🏢 EXL
📍 Maharashtra