Position:
Senior Data Engineer

Job Description:
Job Title: Senior Data Engineer
Location : Pune/Ahmedabad/Indore
Role Summary
Build and operate scalable, reliable data pipelines on Azure. Develop batch and streaming ingestion, transform data using Databricks (PySpark/SQL), enforce data quality, and publish curated datasets for analytics and ML.

Key Responsibilities
Design, build, and maintain ETL/ELT pipelines in Azure Data Factory and Databricks across Bronze → Silver → Gold layers.
Implement Delta Lake best practices (ACID, schema evolution, MERGE/upsert, time travel, Z-ORDER).
Write performant PySpark and SQL; tune jobs (partitioning, caching, join strategies).
Create reusable components; manage code in Git; contribute to CI/CD pipelines (Azure DevOps/GitHub Actions/Jenkins).
Apply data quality checks (Excellent Expectations or custom validations), monitoring, drift detection, and alerting.
Model data for analytics (star/dimensional); publish to Synapse/Snowflake/SQL Server.
Uphold governance and security (Purview/Unity Catalog lineage, RBAC, tagging,



encryption, PII handling).
Author documentation/runbooks; support production incidents and root-cause analysis; suggest cost/performance improvements.

Must-Have (Mandatory)
Data Engineering & Pipelines
Hands-on experience building production pipelines with Azure Data Factory and Databricks (PySpark/SQL).
Working knowledge of Medallion Architecture and Delta Lake (schema evolution, ACID).
Programming & Automation
Solid Python (pandas/PySpark) and SQL.
Practical Git workflow; experience integrating pipelines into CI/CD (Azure DevOps/GitHub Actions/Jenkins).
Familiarity with packaging reusable code (e.g., Python wheels) and configuration-driven jobs.
Data Modeling & Warehousing
Solid grasp of dimensional modeling/star schemas; experience with Synapse, Snowflake, or SQL Server.
Data Quality & Monitoring
Implemented validation checks and alerts; exposure to drift detection and pipeli

📌 Senior Data Engineer Ahmedabad
🏢 Arrow
📍 Ahmedabad