Job Summary:

We are seeking an experienced Azure Databricks Engineer with solid expertise in Azure Data Engineering technologies. The candidate will be responsible for designing, developing, and optimizing scalable data pipelines and data lake solutions using Azure Databricks, PySpark, Azure Data Factory, ADLS Gen2, and Azure Synapse Analytics.

Key Responsibilities:

- Design and implement ETL/ELT pipelines using Azure Databricks and PySpark.
- Develop scalable data ingestion, transformation, and processing solutions.
- Work with structured and unstructured data from various sources such as ADLS Gen2, SQL Databases, APIs, and cloud platforms.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Build and automate workflows using Azure Data Factory (ADF).
- Implement data quality checks, monitoring, logging, and alerting frameworks.
- Collaborate with data architects, analysts, and business stakeholders to gather requirements.




- Participate in code reviews and follow software engineering best practices.
- Ensure data governance, security, and compliance standards are met.

Mandatory Skills:

- Azure Databricks
- PySpark
- Apache Spark
- Azure Data Factory (ADF)
- Azure Data Lake Storage Gen2 (ADLS Gen2)
- Azure Synapse Analytics
- Python
- SQL
- ETL/ELT Development
- Data Warehousing
- Data Lake Architecture
- Spark Performance Tuning

Preferred Skills:

- Delta Lake
- Medallion Architecture
- Azure DevOps / CI-CD
- Unity Catalog
- Data Quality & Monitoring Frameworks

Required Skills
5+ years of experience in Apache Spark / PySpark.
5+ years working with Azure Databricks and Azure Data Services (ADLS Gen2, ADF, Synapse).
Strong understanding of data warehousing, ETL, and data lake architectures

📌 Azure Databricks Engineer (Hyderabad)
🏢 Orcapod Consulting Services
📍 Hyderabad