Role & responsibilities
Design scalable PySpark-based test architectures for ETL/data pipelines, including modular frameworks for batch processing.
Architect end-to-end data validation systems in Hadoop setting for lineage, schema evolution
Lead system design for Hadoop/Hive test settings, including YARN resource management, agile partitioning.
Exposure to Zephyr-Jira-ServiceNow integrated test management systems with experience on API-driven automation.
Design CI/CD test pipelines for PySpark/Hadoop jobs, incorporating artifact management, parallel execution, and blue-green deployments.
Create data quality system designs using PySpark integrated with Hive metadata services.
📌 Data Engineer Gurugram
🏢 Tata Consultancy Services
📍 Gurugram
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.