Job Description Summary

We are seeking an experienced Data Engineer - Databricks to design, build, and operate scalable, high-performance data pipelines on the Databricks Lakehouse Platform. The role involves hands-on development using Apache Spark, Databricks notebooks, Delta Lake, and cloud-native services, along with close collaboration with analytics, AI/ML, and business teams.

Responsibilities
Design, build, and maintain end-to-end data pipelines using Databricks (PySpark / Spark SQL)
Implement batch and incremental data processing using Delta Lake and multi-hop architecture
Develop and optimize Databricks notebooks, jobs, and workflows
Ingest, transform, and curate large-scale structured and semi-structured datasets
Support analytics, reporting, and downstream data consumption use cases
Ensure data quality, reliability, lineage, and governance
Collaborate with data scientists, analysts, and architects on AI/ML workloads
Optimize Spark jobs for performance and cost efficiency
Adhere to enterprise security, access control, and compliance standards
Provide production support and troubleshoot data pipeline issues
Document technical designs, data flows,



and operational runbooks
Mentor junior engineers and contribute to best practices

Mandatory skill sets
4+ years of experience as a Data Engineer with solid Databricks expertise
Hands-on experience with Apache Spark, PySpark, and Spark SQL
Robust knowledge of Delta Lake and Lakehouse architecture
Advanced SQL skills
Experience with ETL/ELT patterns and data warehousing concepts
Exposure to at least one cloud platform (Azure / AWS / GCP)
Understanding of distributed computing concepts
Experience working in Agile teams

Preferred skill sets
Experience with Unity Catalog and data governance
Exposure to Auto Loader or streaming frameworks
CI/CD for data pipelines
Python for data engineering and automation
Databricks certification (Associate / Qualified)

Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their official channels before applying.

📌 In Senior Associate Data Engineer Databricks Data And Analytic Hyderabad
🏢 PwC Service Delivery Center
📍 Hyderabad