Senior Data Engineer Position: Senior Data Engineer : Job Title: Senior Data Engineer Location : Pune/Ahmedabad/Indore Role Summary Build and operate scalable, reliable data pipelines on Azure. Develop batch and streaming ingestion, transform data using Databricks (PySpark/SQL), enforce data quality, and publish curated datasets for analytics and ML.

Key Responsibilities Design, build, and maintain ETL/ELT pipelines in Azure Data Factory and Databricks across Bronze → Silver → Gold layers.

Implement Delta

Lake best practices (ACID, schema evolution, MERGE/upsert, time travel, Z-ORDER). Write performant PySpark and SQL; tune jobs (partitioning, caching, join strategies). Create reusable components; manage code in Git; contribute to CI/CD pipelines (Azure DevOps/GitHub Actions/Jenkins).

Apply data quality checks (Great Expectations or custom validations), monitoring, drift detection, and alerting. Model data for analytics (star/dimensional); publish to Synapse/Snowflake/SQL Server. Uphold governance and security (Purview/Unity Catalog lineage, RBAC, tagging, encryption, PII handling).

Author documentation/runbooks; support production incidents and root-cause analysis; suggest cost/performance improvements. Must-Have (Mandatory) Data Engineering &

- Pipelines Hands-on experience building production pipelines with Azure Data Factory and Databricks (PySpark/SQL). Working knowledge of Medallion Architecture and Delta Lake (schema evolution, ACID). Programming &
- Automation Strong Python (pandas/PySpark) and SQL.

Practical

Git workflow;

experience integrating pipelines into CI/CD (Azure DevOps/GitHub Actions/Jenkins). Familiarity with packaging reusable code (e.g.,



Python wheels) and configuration-driven jobs. Data Modeling &

- Warehousing Solid grasp of dimensional modeling/star schemas;

experience with Synapse, Snowflake, or SQL Server. Data Quality &

- Monitoring Implemented validation checks and alerts; exposure to drift detection and pipeline observability.

Cloud

Platforms (Azure preferred) ADLS Gen2, Key Vault, Databricks, ADF basics (linked services, datasets, triggers), environment promotion. Data Governance &

- Security Experience with metadata/lineage (Purview/Unity Catalog), RBAC, secrets management, and secure data sharing. Understanding of PII/PHI handling and encryption at rest/in transit.

Collaboration

Clear communication, documentation discipline, Agile ways of working, and code reviews.

Databricks Asset

Bundles (DAB) for environment promotion/infra-as-code style deployments. Streaming/real-time: Kafka/Event Hubs

- CDC tools (e.g., Debezium, ADF/Synapse CDC).

MLOps touchpoints: MLflow tracking/registry, feature tables, basic model-inference pipelines. Power BI exposure for publishing curated tables and building operational KPIs.

DataOps practices: automated testing, data contracts, lineage-aware deployments, cost optimization on Azure.

Certifications: Microsoft Certified — Azure Data Engineer Associate (DP-203) or equivalent. 5 years of qualified experience in data engineering (or equivalent project depth). Bachelor’s/Master’s in CS/IT/Engineering or related field (or equivalent practical experience).

Location: IN-GJ-Ahmedabad, India-Ognaj (eInfochips) Time Type: Full time Job Category: Engineering Services Experience Level Senior Level

📌 Senior Data Engineer (Ahmedabad)
🏢 Arrow Electronics India
📍 Ahmedabad