Job Description
We are seeking a skilled Data Engineer to design, build, and optimize Azure-based data pipelines and lakehouse solutions. The ideal candidate will deliver secure, reliable datasets with robust governance, automation, and documentation while collaborating across teams in an Agile environment. This role requires robust expertise in Databricks, Python, Spark, data pipelines, and Azure ecosystem, along with a focus on performance, scalability, and data quality.
n

nKey Responsibilities
n
- n
- Design and develop scalable data pipelines using Azure and Databricksn
- Build and maintain ETL/ELT workflows from diverse data sourcesn
- Experience with Unity Catalog migration and data governance.n
- Develop data transformations using Python, Spark, and Pandasn
- Implement Delta Lake and Medallion architecture (Bronze/Silver/Gold layers)n
- Manage data storage using Azure Data Lake Storage (ADLS)n
- Design and optimize SQL queries and data models (star/dimensional schema)n
- Implement CI/CD pipelines for data workflows using Azure DevOps or GitHubn




- Ensure data quality, validation, monitoring, and observabilityn
- Implement secure data access using Azure Entra ID (SSO/RBAC)n
- Maintain clear documentation for pipelines and data contractsn
- Participate in Agile ceremonies, code reviews, and team collaborationn
- Support and mentor junior team membersn

n

nRequired Skills & Experience
n
- n
- 8+ years of experience in Data Engineeringn
- Strong experience with Azure Databricks (Python, Spark, Pandas)n
- Experience with Azure Data Factory (ADF) for orchestrationn
- Hands-on experience with Delta Lake and Medallion architecturen
- Strong knowledge of Azure Data Lake Storage (ADLS)n
- Experience with Unity Catalog migration and data governancen
- Strong proficiency in SQL and performance optimizationn
- Experience in ETL/ELT and data modeling (star/dimensional schema)n
- Experience with CI/CD tools (Azure DevOps or GitHub Enterprise)n

📌 Data Engineer (Tamil Nadu)
🏢 Kumaran Systems
📍 Tamil Nadu