18 Nov
|
Aditi Consulting
|
Bengaluru
18 Nov
Aditi Consulting
Bengaluru
Summary:
This position involves the identification, validation, and migration of Data Quality (DQ) rules from SAP to Databricks. The role requires a robust understanding of SAP data structures and Databricks platforms to ensure data quality and integrity throughout the migration process.
Job Responsibilities:
- Data Discovery: Identify SAP tables and fields used for existing DQ rules and reports, and locate the corresponding data in Databricks.
- Data Validation: Validate data across all layers (bronze, silver, etc.) in Databricks for completeness, structure, and availability of required fields.
- DQ Rule Migration & BUILD: Select and build a set of existing DQ rules in Databricks as part of the PoC.
- PoC Testing & Reconciliation: Execute the rules in Databricks and compare results with existing DQ results, investigating and resolving differences.
- End-to-End Rule Migration: Following the PoC, migrate/BUILD the agreed As-Is DQ rules to Databricks, including SQL logic, data mappings, rule outputs, and required transformations.
- Data & Rule Validation: Perform end-to-end validation of source data, DQ rules, results,
and defects in Databricks to ensure the migrated process produces expected outcomes.
- DQ Reporting: Support the development of DQ outputs, KPIs, and dashboard reporting from Databricks, including validation of report results.
- Documentation & BAU Handover: Document data mappings, DQ rules, validation results, and migration processes.
Required Skills:
- Proficiency in SQL & Pyspark/Python.
- Experience in Data Quality/Data Validation.
- Experience working with SAP data, preferably master data.
- Good understanding of SAP tables.
- Experience with the Databricks platform.
- Experience in data reconciliation and comparison between different data sources.
- Ability to translate DQ requirements/rules into technical SQL or Python logic.
- Experience in DQ rule testing, defect analysis, and troubleshooting.
Education & Qualifications:
No specific educational qualifications are mentioned, but relevant experience and skills in data quality, SAP, and Databricks are essential.
#AditiConsulting
#26-07319
📌 Data Analyst (Bengaluru)
🏢 Aditi Consulting
📍 Bengaluru