Position:
Senior Data Engineer

Job Description:
Job Title: Senior Data Engineer
Location : Pune/Ahmedabad/Indore
Role Summary
Build and operate scalable, reliable data pipelines on Azure. Develop batch and streaming ingestion, transform data using Databricks (PySpark/SQL), enforce data quality, and publish curated datasets for analytics and ML.

Key Responsibilities
- Design, build, and maintain ETL/ELT pipelines in Azure Data Factory and Databricks across Bronze → Silver → Gold layers.
- Implement Delta Lake best practices (ACID, schema evolution, MERGE/upsert, time travel, Z-ORDER).
- Write performant PySpark and SQL; tune jobs (partitioning, caching, join strategies).
- Create reusable components; manage code in Git; contribute to CI/CD pipelines (Azure DevOps/GitHub Actions/Jenkins).
- Apply data quality checks (Great Expectations or custom validations), monitoring, drift detection, and alerting.
- Model data for analytics (star/dimensional); publish to Synapse/Snowflake/SQL Server.
- Uphold governance and security (Purview/Unity Catalog lineage, RBAC, tagging, encryption,



PII handling).
- Author documentation/runbooks; support production incidents and root-cause analysis; suggest cost/performance improvements.

Must-Have (Mandatory)
- Data Engineering & Pipelines
- Hands-on experience building production pipelines with Azure Data Factory and Databricks (PySpark/SQL).
- Working knowledge of Medallion Architecture and Delta Lake (schema evolution, ACID).
- Programming & Automation
- Solid Python (pandas/PySpark) and SQL.
- Practical Git workflow; experience integrating pipelines into CI/CD (Azure DevOps/GitHub Actions/Jenkins).
- Familiarity with packaging reusable code (e.g., Python wheels) and configuration-driven jobs.
- Data Modeling & Warehousing
- Solid grasp of dimensional modeling/star schemas; experience with Synapse, Snowflake, or SQL Server.
- Data Quality & Monitoring
- Implemented validation checks and alerts; exposure to drift detection and pipeli

📌 Senior Data Engineer (Ahmedabad)
🏢 Arrow
📍 Ahmedabad