Position - Azure Databricks Engineer
Skill Set - Azure Databricks, Pyspark, ADF, SQL,Devops
Experience - 6 - 10 years
Location - Bangalore / Chennai
Notice Period - Immediate Joiner
Job Descriptions :
• Design, develop, and optimize scalable batch and streaming data pipelines using Azure Databricks.
• Develop data processing frameworks using PySpark and Spark SQL.
• Implement Delta Lake-based storage solutions to ensure high reliability and data consistency.
• Build and maintain Bronze, Silver, and Gold data layers within a Lakehouse architecture.
• Ingest data from APIs, databases, Azure storage, and streaming platforms.
• Develop scalable data models supporting analytics and reporting requirements.
• Implement data quality, validation, and reconciliation processes.
• Monitor and optimize Spark workloads for performance and cost efficiency.
• Establish and support data governance practices through Unity Catalog and related solutions.
• Orchestrate and integrate data workflows using Azure Data Factory, Airflow, and Databricks Workflows.
• Collaborate with business and technical stakeholders to gather requirements and deliver enterprise-grade data solutions.