Job Summary

We are currently seeking a Data Engineer - Tester to join our team in Bengaluru, Karntaka (IN-KA), India (IN).

Required Technical Skills
Data Testing Frameworks: Hands-on experience with DataTester to automate data quality checks.
Big Data Processing: Solid proficiency in PySpark and Apache Spark for manipulating and querying large-scale datasets.
Cloud Storage: Deep understanding of AWS S3 storage structures, partitioning strategies, and file formats (Parquet, Delta, Avro).
Testing Methodology: Solid background in functional and non-functional data testing, including data migration and regression testing.
Database & Querying: Advanced SQL skills for querying relational databases and verifying data warehousing solutions.
Version Control: Experience with Git and CI/CD deployment pipelines for automated test execution

Key Responsibilities
Pipeline Validation:



Build automated test suites to validate complex ETL/ELT pipelines using PySpark and DataTester.
Data Quality Assurance: Verify data accuracy, completeness, and consistency across source systems and AWS S3 data lakes.
Test Automation: Design and maintain scalable, reusable data testing frameworks to perform schema validation, null checks, and data reconciliation.
Component Testing: Conduct rigorous unit, integration, and regression testing on distributed data processing applications.
Defect Tracking: Identify, document, and track data anomalies, performance bottlenecks, and pipeline failures.
Collaboration: Partner closely with core Data Engineers and Platform teams to understand data models and implement test-driven development (TDD) practices.

📌 Data Engineer Tester Bengaluru
🏢 NTT DATA Business Solutions
📍 Bengaluru