Senior data engineerIDMC for Data Quality Team IDMC Data Quality Data Engineering consultant
1 Key Responsibilities
Develop IDMC CDI mappings and Taskflows against approved design documents including the DQ evaluation mapping and the DQ orchestration Taskflow nested parallelsequential Iterator pattern Depending on the validated approach implement rule evaluation either as parameterised source SQL with embedded rule expressions CDIonly or as CDQ assets rule specifications maplets and dictionarybased validations integrated into CDI mappings
Build and maintain SQL Server database objects control tables rule library error fact tables staging tables and write productionquality TSQL for SQL tasks within Taskflows audit logging agile SQL assembly watermark logic dimension scoring
Populate and validate the DQ rule library translating quality rules into executable rule definitions across quality dimensions Completeness Validity Consistency Accuracy Uniqueness Timeliness in whichever form the validated approach requires TSQL CASE expressions or CDQ rule specifications
Execute unit testing QA integration testing and support production deployment across all three frameworks
Work in a paired development model with the IPS Lead Developer actively absorbing platform knowledge patterns and IDMC best practices
Participate in sprintend walkthroughs and formal knowledge transfer sessions contribute to implementation documentation
Demonstrate independent capability by engagement end able to add a new source connection register a new DQ rule and debug a failed run using the audit log without IPS support
2 Required Skills And Experiences
Must have
7 years handson data engineering ETL development experience in enterprise environments
Equal and demonstrated proficiency in both IDMC CDI and IDMC CDQ this is a dualcapability role