Req ID: 374353
We are currently seeking a ETL Development Lead to join our team in Bangalore, Karnātaka (IN-KA), India (IN).
Job Title: Offshore ETL/Data Engineer
Role Summary
Key Responsibilities
- Design, develop, and maintain ETL pipelines using Informatica, Azure Data Factory (ADF), and Databricks
- Perform migration of legacy ETL workflows (Informatica) to Databricks using Python/PySpark
- Analyze existing ETL workflows and re-engineer into optimized Spark-based transformations
- Develop data processing and transformation solutions using Python and PySpark
- Build and optimize SQL queries, data models, and transformations
- Schedule and monitor jobs using AutoSys
- Integrate data from multiple sources:
- Relational databases (SQL Server, DB2)
- Files (CSV, XML, JSON)
- Mainframe systems
- Streaming platforms like Kafka
- Perform data validation, reconciliation, and ensure data quality
- Troubleshoot ETL/pipeline failures and optimize performance
- Collaborate closely with onshore teams for development and production support
Work Schedule Requirement
- Offshore support coverage required up to 3:00 PM Central Time (CT) i.e., 2:30AM IST
- Ensure strong overlap for batch monitoring, issue resolution, and critical support
Required Skills:
- Solid experience in Informatica ETL development
- Proven experience in Informatica to Databricks migration
- Strong programming skills in Python and PySpark
- Hands-on experience with Databricks and Azure Data Factory (ADF)
- Proficient in SQL (complex query development and optimization)
- Experience with AutoSys job scheduling
- Experience integrating data from:
- DB / DB2 / Mainframe systems
- Files and streaming platforms (Kafka)
- Solid understanding of ETL re-engineering, transformation logic conversion, and Spark optimization
AI / ML Skills (preferred)
- Basic to intermediate understanding of AI/ML concepts and data pipelines for ML workloads
- Experience using Databricks ML / MLflow / notebooks for model tracking a
📌 Etl Development Lead (Bengaluru)
🏢 NTT
📍 Bengaluru