We are looking for an experienced Databricks MLOps Engineer with 7–10 years of overall experience in building and maintaining scalable data pipelines and machine learning workflows.
The ideal candidate should have strong hands-on experience in Databricks, Apache Spark, PySpark, Delta Lake, MLflow and Unity Catalog, along with expertise in managing machine learning model lifecycles.
Key Responsibilities:
- Build and maintain data pipelines using Databricks, Apache Spark, PySpark and Delta Lake to support machine learning workflows.
- Perform feature engineering and maintain feature tables for model training and scoring.
- Train machine learning models and manage model registration and versioning using MLflow and Unity Catalog.
- Implement model lifecycle management using Unity Catalog, including catalog, schema and model structures.
- Develop and maintain automated scoring pipelines to apply registered models to recent data.
- Monitor model and data quality, track model drift and validate scoring outputs.
- Ensure proper governance of data tables and ML models using Unity Catalog.