Senior Data engineer (India)

Senior Data engineer (India)

18 Nov
|
Kumaran Systems
|
India

18 Nov

Kumaran Systems

India

Key Responsibilities

Data Pipeline Development

• Design and develop scalable data pipelines using Azure Databricks, Python, and PySpark.

• Implement ETL/ELT workflows for structured, semi-structured, and unstructured data.

• Develop and maintain data processing workflows using Delta Lake architecture.

• Ensure efficient data ingestion, transformation, and loading processes.

Data Management & Optimisation

• Optimise performance of data pipelines and processing workloads.

• Ensure data quality, reliability, and consistency across data platforms.

• Write and optimise complex SQL queries for data processing and validation.

• Implement data governance and access controls.

Cloud & Data Platform Integration

• Work with Azure Data Lake for storing and managing large datasets.

• Manage data cataloging and governance using Unity Catalog.

• Support scalable and secure cloud-based data architectures.

Collaboration & Migration Projects

• Work closely with cross-functional teams to deliver data-driven solutions.

• Participate in data migration and modernisation initiatives.

• Support troubleshooting and resolution of data pipeline issues.

• Document data architecture, pipelines, and processes.

Required Skills & Experience





·Must have 7 years of IT experience and at least 5+ years of hands-on experience in Databricks with good communication skills.

·Design and develop scalable data pipelines using Azure Databricks, SCALA, Python, PySpark, and Delta Lake.

·Have a positive understanding of RDBMS and proficiency in writing complex SQL (1st Preference – Oracle / PLSQL).

·Implement ETL/ELT workflows for structured, semi-structured, and unstructured data.

·Optimise performance of data processing and ensure data quality standards.

·Work with Azure Data Lake, Unity Catalogue, and integrate with CI/CD pipelines.

·Experience with orchestration tools like Airflow or Azure Data Factory is a plus.

·Have working experience on a migration project.

·Preferable to have experience in Informatica (although not mandatory)

Relevant Experience Preferred

• Hands-on experience developing data pipelines using Azure Databricks and PySpark.

• Experience implementing ETL/ELT solutions for enterprise data platforms.

• Experience optimizing data processing performance and data workflows.

• Experience working with Azure Data Lake and cloud-based data architectures.

• Experience working on data migration or modernization projects.

📌 Senior Data engineer (India)
🏢 Kumaran Systems
📍 India

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: senior data engineer (india) / india

Subscribe to this job alert:

Get the latest job offers by email for: senior data engineer (india) / india