Senior Platform Engineer (India)

Senior Platform Engineer (India)

18 Nov
|
Insight Global
|
India

18 Nov

Insight Global

India

Job Description Our client is is looking for experienced Data Engineers to join our data engineering team. You will design, build and operate the pipelines that power our enterprise data lake on Microsoft Azure, delivering reliable, high-quality data for operational reporting, analytics and machine learning across our parcel network. Working primarily with Azure Data Factory and Azure Databricks, you will own pipelines from source systems through to curated,
business-ready datasets. The ideal candidate combines robust hands-on engineering skills with a focus on performance, cost efficiency and
data quality, and works effectively with analytics, BI, data science and platform teams. Key Responsibilities:
- Design, develop and maintain scalable batch and streaming pipelines with Azure Data Factory, Azure Databricks and Azure Function Apps,
integrating data from databases, ERP and CRM platforms, files, APIs and event streams.
- Build PySpark and Spark SQL transformations that turn raw data into clean, conformed and trusted datasets, extending the team's
configuration-driven ingestion framework.
- Design Delta Lake tables and lakehouse layers (raw, curated and reporting), with partitioning, clustering and data models aligned to how
the data is consumed.
- Optimize the performance and cost of Spark workloads by tuning joins, file layout and cluster configuration, and eliminate unnecessary
reprocessing.
- Implement monitoring and alerting for pipeline failures, run duration and data freshness, and lead root-cause analysis and resolution of
production incidents.
- Implement data quality checks, schema evolution handling and Unity Catalog access controls to keep data accurate, secure and well
governed.
- Prepare feature tables and training datasets, and support machine learning workflows on Databricks using MLflow and Unity Catalog.
- Deliver curated datasets to analysts, data scientists and Power BI developers, and help optimize semantic models and report refresh
performance.




- Package and deploy jobs and pipelines with Databricks Asset Bundles through Azure DevOps CI/CD, following the team's code review,
testing and documentation standards.
- Mentor junior engineers and contribute to the team's engineering standards Skills and Requirements - 8-12 years in data engineering, with hands-on Azure Databricks and Azure Data Factory experience in production
- Advanced Python, PySpark and SQL, and strong Delta Lake knowledge
- Proven experience tuning Spark workloads for performance and cost
- Experience designing incremental and streaming pipelines and data models
- Hands-on Databricks machine learning experience: MLflow, feature engineering and model training workflows
- Experience with Databricks Asset Bundles and CI/CD in Azure DevOps
- Experience building Azure Function Apps for API and event-driven ingestion
- Production monitoring, alerting and incident management
- Bachelor's degree in a technical field, or equivalent experience
- Spark Structured Streaming, Kafka or Azure Event Hubs
- Lakeflow pipelines and jobs, and Databricks serverless compute
- Unity Catalog administration and Databricks system tables
- Power BI (data modeling, DAX) and Microsoft Fabric
- Terraform or another infrastructure as code tool
- Databricks Data Engineer or Machine Learning certification We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [email protected].

📌 Senior Platform Engineer (India)
🏢 Insight Global
📍 India

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: senior platform engineer (india) / india

Subscribe to this job alert:

Get the latest job offers by email for: senior platform engineer (india) / india