Job Summary

We are seeking an experienced Azure Databricks Engineer with robust expertise in Azure Databricks, PySpark, Python, and SQL to design, develop, and optimize large-scale data engineering solutions. The ideal candidate should have hands-on experience building ETL/ELT pipelines, processing large datasets, and working within the Azure cloud ecosystem.

Key Responsibilities
Design, develop, and maintain scalable data pipelines using Azure Databricks.
Develop and optimize ETL/ELT workflows using PySpark and Python.
Write effective SQL queries for data extraction, transformation, and reporting.
Process structured and unstructured data using Spark.
Integrate Azure Databricks with Azure Data Lake Storage (ADLS), Azure Blob Storage, and other Azure services.
Optimize Spark jobs for performance, scalability, and cost.
Perform data validation, troubleshooting, and performance tuning.
Collaborate with business analysts, data architects, and development teams to understand business requirements.




Follow Agile/Scrum methodologies and participate in sprint planning and code reviews.
Ensure data quality, security, and governance standards are maintained.

Mandatory Skills
Azure Databricks
PySpark
Python
SQL
Robust understanding of Spark architecture and distributed data processing
Experience building ETL/ELT pipelines
Data modeling and performance optimization
Git version control
Agile/Scrum methodology

Good to Have
Azure Data Factory (ADF)
Azure Synapse Analytics
Azure Data Lake Storage (ADLS Gen2)
Delta Lake
Azure DevOps
CI/CD pipelines
Power BI
Kafka/Event Hubs
Snowflake
Azure Key Vault
Data Warehousing concepts

Required Qualifications
Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
715 years of IT experience with strong hands-on data engineering expertise.
Excellent analytical, debugging, and problem-solving skills.
Strong communication and stakeholder management skills.

📌 Azure Data Engineer Bengaluru
🏢 Novastrid IT Ventures
📍 Bengaluru