Line of Service Advisory Industry Sector Not Applicable Specialism Microsoft Management Level Manager Summary At PwC our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients They play a crucial role in transforming raw data into actionable insights enabling informed decision-making and driving business growth Those in data and automation at PwC will focus on automating data internally using automation tools or software to streamline data-related processes within the organisation Your work will involve automating data collection data analysis data visualisation or any other tasks that involve handling and processing data Why PWC At PwC you will be part of a vibrant community of solvers that leads with trust and creates distinctive outcomes for our clients and communities This purpose-led and values-driven work powered by technology in an environment that drives innovation will enable you to make a tangible impact in the real world We reward your contributions support your wellbeing and offer inclusive advantages flexibility programmes and mentorship that will help you thrive in work and life Together we grow learn care collaborate and create a future of infinite experiences for each other Learn more At PwC we believe in providing equal employment opportunities without any discrimination on the grounds of gender ethnic background age disability marital status sexual orientation pregnancy gender identity or expression religion or other beliefs perceived differences and status protected by law We strive to create an environment where each one of our people can bring their true selves and contribute to their personal growth and the firm s growth To enable this we have zero tolerance for any discrimination and harassment based on the above considerations We are looking for a Solution Architect with deep expertise in Databricks Data Engineering and Python-based software development to design engineer and deploy secure scalable data platforms on Azure Databricks The role demands hands-on experience in Spark-based transformations CI CD automation and Databricks asset bundle deployment with a strong grasp of production-grade governance practices Responsibilities Databricks Data Engineering o Build and optimize robust ETL ELT pipelines using PySpark and Delta Lake o Design bronze-silver-gold architecture schema evolution and partitioning strategies o Manage Databricks Unity Catalog lineage and ACLs for data governance o Implement data quality checks validation layers and performance tuning for large-scale datasets Python Software Engineering o Develop reusable modular and testable Python libraries for data ingestion transformation and utilities o Enforce code quality through unit testing linters and error handling standards o Package code into custom wheels support dependency management and ensure compatibility across environments Databricks Deployment CI CD o Automate workspace deployment using Databricks Asset Bundles DAB and workspace files o Configure CI CD pipelines via Azure DevOps GitHub Actions and Databricks Repos o Implement deployment promotion workflows across dev test and prod o Ensure audit logging run tracking and change traceability for governance and compliance o Manage secrets key vault integrations and network security private link VNet injection Skill sets o 5 years on Azure Data Stack ADF ADLS Gen2 Synapse Event Hub o 3 years hands-on with Databricks Spark Delta DLT Unity Catalog o Strong command of Python with production-grade software engineering practices Deep experience in CI CD for Databricks DAB CLI Repos DevOps GitHub Actions o Strong knowledge of IaC Terraform Bicep for deploying data infrastructure o Hands-on experience with Git-based workflows release versioning rollback production environments - Integrate with various databases e g Cosmos DB and message queues e g Kafka eventhub for seamless backend operations - Experience in one or more Python frameworks Django Flask - Develop and maintain unit and integration tests using frameworks like pytest and unit test to ensure code quality and reliability Awareness of observability monitoring and security practices Certifications Credentials o Databricks Certified Data Engineer Associate Professional o Microsoft Certified Azure Solutions Architect AZ-305 o Microsoft Certified Azure Data Engineer DP-203 o Terraform Associate optional Mandatory skill sets Data Engineer Preferred skill sets Data Engineer Years of experience required 6-10 yrs Education qualification Bachelor s or Master s degree in Computer Science Engineering or related field Education if blank degree and or field of study not specified Degrees Field of Study required Bachelor of Technology Bachelor of Engineering Degrees Field of Study preferred Certifications if blank certifications not specified Required Skills Data Engineering Optional Skills Accepting Feedback Accepting Feedback Active Listening Agile Methodology Alteryx Automation Platform Analytical Thinking Automation Automation Framework Design and Development Automation Programming Automation Solutions Automation System Efficiency Business Analysis Business Performance Management Business Process Automation BPA Business Transformation C Programming Language Coaching and Feedback Communication Configuration Management CM Continuous Process Improvement Creativity Daily Scrum Data Analytics Data Architecture Data-Driven Insights 39 more Desired Languages If blank desired languages not specified Travel Requirements Available for Work Visa Sponsorship Government Clearance Required Job Posting End Date

📌 In Manager Data Engineer Application Technology Advisory Kolkata
🏢 PwC
📍 Kolkata