Job Summary

We are currently seeking a Data Engineer - Tester to join our team in Bengaluru, Karntaka (IN-KA), India (IN).

Required Technical Skills

- Data Testing Frameworks: Hands-on experience with DataTester to automate data quality checks.
- Big Data Processing: Solid proficiency in PySpark and Apache Spark for manipulating and querying large-scale datasets.
- Cloud Storage: Deep understanding of AWS S3 storage structures, partitioning strategies, and file formats (Parquet, Delta, Avro).
- Testing Methodology: Solid background in functional and non-functional data testing, including data migration and regression testing.
- Database & Querying: Advanced SQL skills for querying relational databases and verifying data warehousing solutions.
- Version Control: Experience with Git and CI/CD deployment pipelines for automated test execution

Key Responsibilities
- Pipeline Validation:



Build automated test suites to validate complex ETL/ELT pipelines using PySpark and DataTester.
- Data Quality Assurance: Verify data accuracy, completeness, and consistency across source systems and AWS S3 data lakes.
- Test Automation: Design and maintain scalable, reusable data testing frameworks to perform schema validation, null checks, and data reconciliation.
- Component Testing: Conduct rigorous unit, integration, and regression testing on distributed data processing applications.
- Defect Tracking: Identify, document, and track data anomalies, performance bottlenecks, and pipeline failures.
- Collaboration: Partner closely with core Data Engineers and Platform teams to understand data models and implement test-driven development (TDD) practices.

📌 Data Engineer - Tester (Bengaluru)
🏢 NTT DATA Business Solutions
📍 Bengaluru