Azure Databricks Architect (Pune)

Azure Databricks Architect (Pune)

17 Nov
|
Mississippi Consultants Pune
|
Pune

17 Nov

Mississippi Consultants Pune

Pune

Role & responsibilities

We are seeking an experienced and detail-oriented Azure Databricks Data Engineer to join our growing data team. In this role, you will be responsible for designing, developing, implementing, and maintaining high-performance, scalable data pipelines and solutions within the Microsoft Azure ecosystem. You will collaborate closely with cross-functional teams to translate business requirements into efficient ETL/ELT workflows, ensuring optimal data quality, performance, and cost-efficiency.

Key Responsibilities

- Data Integration & Pipelines: Design, build, and deploy end-to-end scalable data pipelines and robust ETL/ELT workflows using Azure Databricks, PySpark, Spark SQL, and Azure Data Factory (ADF).
- Architecture Implementation: Build and optimize lakehouse architectures leveraging Delta Lake (Medallion architecture: Bronze, Silver, Gold layers) to manage structured, semi-structured (JSON, XML), and unstructured data.
- Performance Tuning: Write highly efficient PySpark and SQL code. Perform partition tuning, caching optimizations, and cluster scaling to minimize execution runtimes and operational cloud costs.
- Data Governance & Security: Set up, configure, and govern data workspaces using Unity Catalog. Secure sensitive data using Azure Key Vault-backed secrets and implement role-based access controls (RBAC).
- CI/CD & DevOps:



Implement modern DevOps practices, including version control with Git, and automate deployments through CI/CD pipelines (e.g., Azure DevOps or GitHub Actions).
- Monitoring & Troubleshooting: Set up proactive monitoring, logging, and alerting systems via Azure Monitor or Databricks Workflows to diagnose and resolve complex data pipeline failures.

Required Technical Skills & Qualifications

- Education: Bachelors degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
- Big Data & Cloud: 5+ years of hands-on data engineering experience, with at least 3 years explicitly focused on Azure Databricks, Apache Spark, and Azure Data Lake Storage (ADLS Gen2).
- Programming Languages: Proficiency in Python / PySpark and advanced SQL for complex transformations, data modeling, and performance tuning.
- Data Tools: Direct experience working with Delta Lake and Azure Synapse Analytics.
- Methods: Solid understanding of dimensional data modeling, data warehousing principles, and data quality validation frameworks.
- Agile Ecosystem: Experience working in an Agile/Scrum workplace with cross-functional data scientists, analysts, and business stakeholders

Contact-Soniya

[email protected]

📌 Azure Databricks Architect (Pune)
🏢 Mississippi Consultants Pune
📍 Pune

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: azure databricks architect (pune) / pune

Subscribe to this job alert:

Get the latest job offers by email for: azure databricks architect (pune) / pune