Job Summary

Position: Senior Databricks / Machine Learning Engineer. Experience: 8+ Years Mandatory. Support Window: 6:00 PM - 10:00 AM. Assignment: 3 Hours Daily. Mode: Remote / Part-Time. Focus: Databricks + Machine Learning.

Role Overview

We are looking for an experienced Senior Databricks / Machine Learning Engineer with 8+ years of experience for a part-time job support assignment.

The ideal candidate must have strong hands-on experience in Azure, Azure Databricks, Apache Spark, PySpark and Machine Learning, with experience building data ingestion, processing, ML experimentation and deployment workflows.

Candidates must be available for a 3-hour assignment within the 6:00 PM - 10:00 AM support window.

Must-Have Technologies

- Cloud: Microsoft Azure
- Data Ingestion: Azure Synapse Pipelines
- Storage: Azure Data Lake Storage (ADLS)
- Data Processing: Azure Databricks; Apache Spark; PySpark
- Data Formats: Delta Lake / Parquet
- Machine Learning: Python; Keras; Machine Learning model development
- Hyperparameter Tuning: Optuna
- Experiment Tracking: MLflow
- Model Registry: MLflow Model Registry
- Workflow Orchestration: Databricks Jobs; YAML Configurations
- Source Control: GitHub
- Artifact Management: JFrog Artifactory
- Documentation: Confluence
- Operational / AI Analysis: Genie Agents; Future custom-agent ecosystem
- Deployment: Plant Servers
- Supporting Systems: DTR; BOPH / Equipment Metadata; Plant Telemetry Systems

Key Responsibilities

- Develop and support Azure Databricks data and ML solutions.




- Build scalable data processing pipelines using Spark/PySpark.
- Work with Azure Synapse Pipelines for data ingestion.
- Process and manage data in ADLS using Delta/Parquet formats.
- Develop ML solutions using Python and Keras.
- Implement hyperparameter optimization using Optuna.
- Manage ML experiments and model lifecycle using MLflow.
- Maintain MLflow Model Registry.
- Configure and manage Databricks Jobs and YAML-based workflows.
- Use GitHub for source control and JFrog Artifactory for artifact management.
- Support deployment of models/solutions to plant servers.
- Work with plant telemetry, equipment metadata, DTR and BOPH systems.
- Support operational analysis involving Genie agents and future custom-agent ecosystems.
- Troubleshoot data, ML and deployment issues.
- Maintain technical documentation in Confluence.

Qualifications

- 8+ years overall IT experience
- Strong Azure Databricks experience
- Strong Machine Learning experience
- Strong Python + PySpark + Apache Spark
- Azure Data Lake Storage experience
- Azure Synapse Pipelines experience
- Keras / ML model development experience
- Optuna / hyperparameter tuning experience
- MLflow + MLflow Model Registry experience
- Databricks Jobs + YAML experience
- GitHub experience
- Experience with Delta / Parquet
- Positive understanding of ML lifecycle and deployment
- Experience working with production/operational systems
- Available for 3 hours daily
- Available within 6 PM - 10 AM support window

📌 Senior Databricks / Machine Learning Engineer (Bengaluru)
🏢 Supreme Consulting
📍 Bengaluru