This is Hybrid role and can sit either of locations Bengaluru, Hyderabad, Chennai and Nagpur
Job Requirements and Responsibilities for Azure Databricks Architect:
- Design, develop, and maintain scalable data pipelines using Azure Databricks.
- Build high-performance ETL/ELT pipelines using PySpark and Apache Spark.
- Implement Declarative Pipelines and optimize workflows for reliability and scalability.
- Design and manage Delta Lake architecture for effective data storage and processing.
- Develop robust batch and streaming data processing solutions.
- Create and optimize SQL queries for large-scale data processing.
- Perform performance tuning of Spark jobs and Databricks clusters.
- Implement data governance using Unity Catalog.
- Design enterprise-level data models supporting analytics and reporting.
- Integrate CI/CD pipelines for automated deployment and testing.
- Collaborate with Data Scientists and AI teams to enable ML/AI workloads.
- Ensure data quality, security, monitoring, and best practices across the platform.
- Databricks Certification Mandatory