Job Description: Data Engineer – Data Pipelines & Governance

Location: Chennai (Preferred) / Pune / Hyderabad / Bangalore

Contract Duration: (6 Months)

Shift: UK Shift (2:00 PM – 11:00 PM)

Overview

We are seeking a hands-on Data Engineer to develop, optimize, and maintain automated data pipelines supporting data governance and analytics initiatives. This role will focus on building production-ready workflows for ingestion, transformation, quality checks, lineage capture, access auditing, cost usage analysis, retention tracking, and metadata integration, primarily using Azure Databricks, Azure Data Lake, and MS Purview.

Experience: 5+ years in data engineering, with robust Azure and Databricks experience

Key Responsibilities
Pipeline Development – Design, build, and deploy robust ETL/ELT pipelines in Databricks (Py Spark, SQL, Delta Lake) to ingest, transform, and curate governance and operational metadata from multiple sources landed in Databricks. Granular Data Quality Capture – Implement profiling logic to capture issue-level metadata (source table, column,



timestamp, severity, rule type) to support drill-down from dashboards into specific records and enable targeted remediation. Governance Metrics Automation – Develop data pipelines to generate metrics for dashboards covering data quality, lineage, job monitoring, access & permissions, query cost, usage & consumption, retention & lifecycle, policy enforcement, sensitive data mapping, and governance KPIs. MS Purview Integration – Automate asset onboarding, metadata enrichment, classification tagging, and lineage extraction for integration into governance reporting. Data Retention & Policy Enforcement – Implement logic for retention tracking and policy compliance monitoring (masking, RLS, exceptions). Job & Query Monitoring – Build pipelines to track job performance, SLA adherence, and query costs for cost and performance optimization. Metadata Storage & Optimization – Maintain curated Delta tables for gover

📌 Data Engineer (Madhya Pradesh)
🏢 Dexian
📍 Madhya Pradesh