Senior Data Engineer – Databricks / Azure

Experience: 5+ Years
Employment Type: Full-Time
Work Mode: Remote
We are looking for an experienced Data Engineer with strong hands-on expertise in Databricks, PySpark, Delta Lake, and Azure Data Engineering to design and develop scalable data pipelines and cloud-based data solutions.

Primary Skills

- Databricks
- PySpark
- Delta Lake
- Azure Data Factory (ADF)
- Logic Apps
- Apache Airflow

Secondary Skills

- Python
- SQL
- Azure Data Lake Storage Gen2 (ADLS Gen2)
- CI/CD
- MLflow
- Data Observability
- Azure Cloud

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
- Build and optimize data processing solutions using Delta Lake.
- Develop and manage ETL/ELT workflows using Azure Data Factory and Airflow.
- Integrate Azure services including ADLS Gen2 and Logic Apps into data solutions.
- Write productive Python and SQL code for data transformation and processing.
- Implement data quality, monitoring, and data observability practices.
- Optimize Spark jobs,



Delta tables, and data pipelines for performance and scalability.
- Implement CI/CD pipelines for data engineering workflows and deployments.
- Collaborate with data scientists and engineering teams on MLflow and machine learning workflows.
- Troubleshoot production data pipelines and ensure reliability and availability.

Required Qualifications

- 5+ years of experience in Data Engineering.
- Strong hands-on experience with Databricks, PySpark, and Delta Lake.
- Good experience with Azure Data Factory, ADLS Gen2, and Azure Cloud.
- Strong programming skills in Python and SQL.
- Experience with workflow orchestration using Airflow.
- Understanding of CI/CD, data quality, monitoring, and cloud-based data platforms.

Preferred

Experience with MLflow, Logic Apps, Data Observability, and enterprise-scale Azure data platforms will be an added advantage.

Work Location: Remote

📌 Data Engineer (India)
🏢 WebSenor InfoTech
📍 India