13 Aug
|
Talent Skeet
|
Bengaluru
13 Aug
Talent Skeet
Bengaluru
Role & responsibilities • Assess complex Informatica workflows including sessions, mappings, and parameterizations and translate them into equivalent Python EL scripts and dbt transformation models. • Develop and maintain Python EL pipelines to land high-volume data, including billion-row tables, into the target warehouse using SQLAlchemy, cx_Oracle, pyodbc, and bulk export tools such as Oracle Data Pump and SQL Server BCP. • Design and develop dbt transformation models based on Informatica mapping logic, incorporating dbt best practices including model layering, macros, incremental strategies, and snapshot patterns. • Develop dbt-native tests as well as custom Python unit tests to validate transformation correctness and data quality. • Develop Airflow DAGs to orchestrate Python EL and dbt scripts end-to-end, producing output that is functionally equivalent to the source Informatica workflows. • Contribute to GitLab CI/CD pipeline for dbt and Airflow code, including lint gates, automated testing, and deployment to shared NAS.
• Perform peer code reviews and provide constructive technical feedback to fellow engineers. • Troubleshoot performance issues and data discrepancies during SIT and UAT, including row-count reconciliation between source Oracle/SQL Server systems and the target warehouse. • Contribute to technical documentation, runbooks, and handover materials.
Preferred candidate profile
• 5+ years of Informatica PowerCenter experience - mappings, sessions, workflows, and parameterization • 5+ years of Python experience, including production use of parallelism and batch data processing • 3+ years of hands-on DBT Core experience - model layering, macros, tests, incremental materializations • 3+ years of Airflow experience - DAG authoring, task dependencies, and incorporation with scheduling tool • 5+ years of SQL experience - complex joins, window functions, aggregations across large datasets
📌 Data Migration Engineer (Bengaluru)
🏢 Talent Skeet
📍 Bengaluru