Location : NoidaWork Mode : Work From OfficeExperience : 9-12 YearsRole Overview : We are looking for an experienced Senior Data Engineer with strong hands-on expertise in Azure Databricks, PySpark, Python and SQL to design, develop and optimize scalable enterprise data engineering solutions.The ideal candidate should have extensive experience in building robust ETL/ELT pipelines, processing large-scale datasets, implementing modern Lakehouse architectures and optimizing data platforms for performance, scalability and cost efficiency.The candidate will also drive Databricks serverless migration initiatives, CI/CD implementation and engineering best practices.Key Responsibilities : - Design, develop and maintain scalable data pipelines using Azure Databricks, PySpark, Python and SQL.- Build and optimize ETL/ELT workflows for structured and semi-structured data.- Develop and manage data pipelines using Apache Spark, Spark SQL and Delta Lake.- Lead migration of existing Databricks workloads to serverless compute architecture.- Implement and maintain CI/CD pipelines using GitHub Actions or equivalent tools.- Manage code repositories, monitor pull requests, conduct code reviews and ensure deployment quality.- Optimize Spark jobs, data processing workflows and Databricks workloads for performance and cost efficiency.- Implement data governance, security, access control and compliance best practices.- Develop modular, reusable and maintainable code; refactor existing pipelines where required.- Work closely with cross-functional teams to understand business requirements and deliver reliable data solutions.- Support automated testing, data validation, monitoring and production troubleshooting.Mandatory Technical Skills : - Azure Databricks :



Minimum 2+ years of hands-on experience.- Data Engineering : 8+ years of relevant experience in Data Engineering, ETL/ELT and large-scale data processing.- Programming : Strong hands-on expertise in Python and PySpark.- Apache Spark : Strong understanding of Spark architecture, Spark SQL and performance optimization.- SQL : Advanced SQL skills, including complex queries and data transformations.- ETL/ELT : Experience designing and implementing enterprise-grade data pipelines.- Data Architecture : Strong understanding of Delta Lake, Lakehouse architecture and data modelling.- Orchestration : Experience with Apache Airflow and/or equivalent orchestration tools.- CI/CD : Hands-on experience with GitHub Actions, Azure DevOps or similar CI/CD platforms.Preferred / Good-to-Have Skills : - Delta Live Tables (DLT).- Unity Catalog and data governance.- Delta Sharing.- Databricks SQL Warehouse, including serverless SQL Warehouse.- Serverless compute architecture and migration.- dbt or similar data transformation frameworks.- Cloud security, access management and compliance.- Experience with financial data, pricing logic or complex business calculations.- Exposure to AI/ML-based data solutions.- Experience leading technical teams, mentoring engineers and conducting code reviews.Ideal Candidate Profile : - Strong hands-on engineering background rather than only project coordination or team management.- Proven experience developing production-grade Databricks and PySpark solutions.- Experience handling large-scale data migration and transformation projects.- Solid problem-solving, analytical and communication skills.- Ability to independently drive technical initiatives and collaborate with stakeholders. (ref:hirist.tech)

📌 Senior Data Engineer - Azure Databricks (Noida)
🏢 wowJobs
📍 Noida