Position Summary:
The Data Engineer will design, build, and optimize scalable data pipelines and curated data products within a modern Lakehouse architecture (Databricks). This role is responsible for delivering high-quality, business-ready datasets (gold layer) that power enterprise analytics and Power BI reporting. The ideal candidate has deep Databricks experience, solid data modeling skills, and a focus on building reliable, testable, and governed data solutions. Exposure to AI-driven development or AI agents is a plus.

Work You’ll Do:
Data Engineering & Pipeline Development
· Design, develop, and maintain scalable data pipelines using Databricks and Delta Lake
· Build and manage transformations across bronze, silver, and gold layers
· Optimize processing for performance, reliability, and cost
· Integrate data from ERP, CRM, APIs, and other enterprise systems
Data Modeling & Business Logic
· Develop gold-layer datasets aligned to standardized business definitions
· Translate business requirements into reusable data models
· Ensure consistency of core metrics across reporting




· Align Databricks outputs with Power BI semantic models
Data Quality, Testing & Reliability
· Implement automated data quality checks and validation rules
· Build testable, production-ready pipelines
· Support impact analysis using lineage tools
· Participate in CI/CD and deployment processes
Performance Optimization & Operations
· Monitor and optimize pipeline performance
· Troubleshoot issues across environments
· Ensure data consistency between dev, test, and prod
· Support high-volume data workloads
AI & Automation (Preferred)
· Use AI-assisted tools for development (e.g., Copilot, Databricks Agents)
· Explore AI agents for testing, lineage analysis, and optimization
· Contribute to AI-driven engineering practices
Collaboration & Stakeholder Engagement
· Partner with BI and business teams
· Support governance and cataloging efforts
· Document data models and pipelines

Basic Qualifica

📌 Data Engineer – Databricks (Coimbatore)
🏢 Hillenbrand
📍 Coimbatore