Job Title: Databricks QA / Data Reconciliation Engineer

Role Overview

We are looking for a Data QA / Reconciliation Engineer with hands-on experience in Databricks, SQL, and data validation. The candidate will be responsible for validating data pipelines, reconciling data between source and target systems, verifying transformation logic, and ensuring data accuracy and consistency across data processing layers.

Key Responsibilities

- Perform end-to-end data validation and reconciliation across source systems, staging tables, and target datasets in Databricks.
- Validate ETL/ELT pipelines, data transformations, and source-to-target mappings.
- Write complex SQL queries to verify record counts, totals, aggregations, data completeness, and accuracy.
- Identify and investigate data mismatches, missing records, duplicate records, and transformation errors.
- Perform data quality checks across Databricks Delta tables and Bronze, Silver, and Gold data layers.
- Design and execute test cases for data integration, functional testing, regression testing, and end-to-end validation.
- Use Python/PySpark to automate data validation and reconciliation wherever required.
- Validate schema changes, business rules, derived fields, and incremental data loads.




- Log defects, perform root cause analysis, and coordinate with data engineering teams to resolve issues.
- Maintain test scripts, reconciliation reports, test evidence, and data quality documentation.

Required Technical Skills

- Strong hands-on experience in SQL, including complex joins, aggregations, subqueries, and data comparison.
- Experience in ETL testing, data warehouse testing, data validation, and reconciliation.
- Practical experience with Databricks and Delta Lake.
- Understanding of data pipelines, data transformations, and source-to-target mapping.
- Experience in test case design, defect tracking, and regression testing.
- Valuable analytical and problem-solving skills.

Preferred Skills

- Python and PySpark for test automation and data validation.
- Experience with AWS data services and cloud-based data platforms.
- Familiarity with data warehouse concepts, data modelling, and incremental data processing.
- Exposure to automated data quality frameworks and CI/CD processes.
- Experience working in Agile teams using tools such as Jira or similar test management platforms.

📌 Databricks QA / Data Reconciliation Engineer (Hyderabad)
🏢 Blue Cloud Softech Solutions
📍 Hyderabad