Role Overview

We are looking for a Senior Azure Data Engineer with strong expertise in Databricks and Azure data services. The role involves designing, building, and optimizing scalable, secure, and high-performance data platforms, leveraging modern Lakehouse architecture and real-time data processing capabilities.

Key Skills

- Databricks : Notebooks (Python, SQL), SQL Editor, Unity Catalog, Delta Lake, Databricks Notification Alerts, Genie Spaces, and performance optimization.
- Azure Data Factory (ADF) for ETL/ELT orchestration.
- Advanced Python, PySpark, and SQL skills with robust understanding of Spark performance tuning.
- Azure Ecosystem : Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, Azure SQL Database, and Azure Event Hub.
- Real-time data ingestion : Structured Streaming, Kafka, Event Hubs, and Auto Loader.
- Architecture : Spark Declarative Pipelines (SDP), Delta Live Tables (DLT), and Medallion Architecture.
- Governance : Data modeling, data warehousing, governance, lineage, and access control.
- DevOps : GitHub, GitHub Actions, Azure DevOps, and CI/CD practices.
- Familiarity with Databricks Asset Bundles (DAB) and Lakeflow Connect.

Responsibilities





- Design and develop scalable data pipelines using Databricks and ADF.
- Build and maintain Delta Lake tables and Lakehouse solutions for analytics and reporting.
- Implement governance and security using Unity Catalog.
- Optimize Databricks jobs, Spark workloads, and SQL queries for performance and cost efficiency.
- Develop and support real-time and batch data processing solutions.
- Configure Databricks Notification Alerts, SQL dashboards, and Genie Spaces to improve monitoring, self-service analytics, and business user adoption.
- Monitor, troubleshoot, and improve data pipeline reliability and quality.
- Collaborate with stakeholders to deliver business-focused data solutions.
- Mentor junior engineers and promote data engineering best practices.

Preferred Qualifications

- 5+ years of Azure Data Engineering experience.
- Experience with large-scale distributed data processing systems.
- Databricks and/or Azure Data Engineering certifications preferred.
- Strong communication, collaboration, and problem-solving skills.

(ref:hirist.tech)

📌 USEReady Technology - Senior Data Engineer - Databricks/Azure (Kolkata)
🏢 USEReady
📍 Kolkata