ETL Databricks Testing

Required Technical Skill Set :

Robust experience in ETL testing and data warehouse concepts.

• Hands-on expertise in Databricks (including notebooks, clusters, and jobs).

• Proficiency in SQL for data validation and analysis.

• Experience with big data technologies (Spark, PySpark, Delta Lake).

• Familiarity with cloud platforms (Azure, AWS, or GCP) and their data services.

• Knowledge of data modeling, data quality frameworks, and BI tools.

• Strong analytical and problem-solving skills.

• Excellent communication and documentation abilities.

Must Have:
ETL / Data Warehouse Testing: o Strong experience in DWH concepts (Facts, Dimensions, Star/Snowflake schema) o Hands-on data validation across large datasets
Databricks: o Experience validating data pipelines built on Databricks o Working knowledge of notebooks and Delta Lake concepts
SQL: o Solid proficiency in writing complex SQL queries (joins, subqueries, aggregations, window functions)
Ability to perform large-volume data validation and reconciliation




Python (Basic to Intermediate): o Ability to read/write easy Python scripts for data validation, automation support, or log analysis

Responsibility :

1 Validate end to end ETL/data pipeline workflows from source systems to target data warehouse layers

2 Perform data reconciliation, data quality checks, and transformation validation across Bronze / Silver / Gold layers (or equivalent)

3 Execute Databricks-based data validations using SQL, notebooks, and result comparisons

4 Design and execute complex SQL queries for data profiling, completeness, accuracy, and referential integrity checks

5 Validate Delta tables, fact/dimension tables, aggregates, and historical load logic

6 Perform job failure analysis, data discrepancy RCA, and support defect triage

7 Collaborate closely with Data Engineering, Analytics, and QA teams during releases

8 Ensure test coverage for incremental loads, CDC, reprocessing, and batch schedules

📌 Databricks Etl Testing 14th July Tuesday Virtual Drive Hyderabad
🏢 Tata Consultancy Services
📍 Hyderabad