Job Description Azure Databricks (ADB) & PySpark

Responsibilities
Develop and maintain data pipelines using Azure Databricks and PySpark.
Design, build, and optimize ETL/ELT processes for large-scale data processing.
Work with structured and unstructured data from multiple sources.
Develop data transformation and validation logic using PySpark.
Integrate Azure Databricks with Azure Data Factory and other Azure services.
Optimize Spark jobs for performance and scalability.
Troubleshoot and resolve data processing issues.
Collaborate with business stakeholders, analysts, and data engineering teams.
Follow best practices for coding, testing, and deployment.

Required Skills
Robust hands-on experience with Azure Databricks and PySpark.




Valuable knowledge of SQL and data warehousing concepts.
Experience with Azure Data Factory (ADF).
Understanding of Spark architecture and performance tuning.
Experience working with Delta Lake and Data Lake concepts.
Knowledge of Git and CI/CD processes.
Strong analytical and problem-solving skills.

Valuable to Have
Experience with Azure Synapse Analytics.
Exposure to Power BI.
Knowledge of Python and cloud-based data engineering practices.
Experience in Banking, Finance, or Retail domains.

📌 Azure Data Engineer Hyderabad
🏢 Hexaware Technologies
📍 Hyderabad