Note: - This is 6 Months Contractual role based on projects chances of extension is 90% for 12 months further with 100% remote opportunity.

We are looking for a highly experienced Azure Databricks Data Engineer with 8+ years of overall experience and strong hands-on expertise in Azure Databricks, PySpark, Databricks Asset Bundles (DAB), Azure DevOps, and CI/CD.

The ideal candidate will be responsible for designing and developing scalable data engineering solutions, implementing robust ETL/ELT pipelines, and automating Databricks deployments across multiple environments using DevOps best practices.

Key Responsibilities

- Design, develop, and maintain enterprise-scale data pipelines using Azure Databricks and PySpark.
- Develop scalable ETL/ELT solutions using PySpark, Spark SQL, and SQL.
- Build and optimize Delta Lake tables and Lakehouse data processing workflows.
- Implement and manage Databricks Asset Bundles (DAB) for packaging, configuration, and deployment of Databricks resources.
- Design and maintain Azure DevOps CI/CD pipelines for Databricks deployments.
- Automate deployment of Databricks notebooks, jobs, workflows, and configurations across Dev, QA, UAT, and Production environments.
- Integrate Databricks development with Git, GitHub, or Azure Repos.
- Implement setting-specific configurations and deployment strategies.
- Work with Azure Data Lake Storage Gen2,



Azure Key Vault, Azure Data Factory, and other Azure services.
- Develop reusable PySpark frameworks and production-ready data processing solutions.
- Perform Spark performance tuning, including optimization of partitions, joins, caching, file sizes, and Delta Lake operations.
- Implement data quality, validation, logging, monitoring, and error-handling frameworks.
- Troubleshoot complex production issues and provide technical solutions.
- Collaborate with architects, data engineers, DevOps teams, and business stakeholders.
- Provide technical guidance and mentoring to junior and mid-level engineers.

Required Skills

Must Have

- 8+ years of overall experience in Data Engineering / Big Data / Cloud Data Engineering.
- Strong hands-on experience with Azure Databricks.
- Strong expertise in PySpark and Apache Spark.
- Hands-on experience with Databricks Asset Bundles (DAB).
- Strong experience with Azure DevOps and CI/CD pipelines.
- Experience automating Databricks deployments across multiple environments.
- Strong knowledge of Delta Lake and Lakehouse architecture.
- Strong SQL skills.
- Experience with Git / GitHub / Azure Repos.
- Experience with Azure Data Lake Storage Gen2 (ADLS Gen2).
- Strong understanding of ETL/ELT concepts and modern data engineering practices.

📌 Senior Data Engineer (India)
🏢 Solvex Solutions
📍 India