Job Title: Senior Data Engineer

Employment Type: Full Time, Permanent

Position Location: Hyderabad

Qualifications: BE/B. Tech/MCA Degree in Computer Science, Engineering, or similar relevant field

Total Experience: 8 to 14 years

Working Model: Work from office

Data Pipeline Development

Design and implement end-to-end batch and streaming data pipelines using Py Spark and Apache Spark. Build and manage data workflows on Databricks, including Delta Lake architecture and medallion (Bronze/Silver/Gold) patterns. Develop and maintain ETL/ELT processes to ingest, transform, and load data from diverse sources. Optimize Spark jobs for performance, cost-efficiency, and reliability at scale.

Cloud & Azure Services

Manage data solutions using Azure services including Azure Data Lake Storage (ADLS Gen2), Azure Event Hubs, and Azure Key Vault. Ensure data security, access control, and compliance within Azure environments. Monitor and troubleshoot cloud infrastructure, pipelines, and data flows.

Data Modeling & SQL

Design and implement dimensional and normalized data models for analytical and operational use cases.



Write complex SQL queries for data transformation, validation, and reporting. Work with structured, semi-structured, and unstructured data across various storage formats (Parquet, Delta, JSON, CSV).

Required Skill

6+ years of hands-on experience in data engineering roles. Strong expertise in Apache Spark and Py Spark for large-scale data processing. Deep experience with Databricks platform — workspace setup, clusters, jobs, Delta Live Tables. Strong experience in building Azure Data Factory Pipeline. Proficiency with Azure cloud services for data engineering workloads. Advanced SQL skills including window functions, CTEs, and query optimization.

Solid

Python programming skills for data engineering, scripting, and automation.

Experience with version control (Git) and CI/CD pipelines for data code deployments. Familiarity with data governance, data lineage, and metadata management.

Positive to have

- Azure Synapse & Azure Event Hub are good to have.

📌 Sr azure data engineer - hyderabad
🏢 JobCrexa
📍 Hyderabad