n
- Design scalable PySpark-based test architectures for ETL/data pipelines, including modular frameworks for batch processing.
n
- Architect end-to-end data validation systems in Hadoop workplace for lineage, schema evolution
n
- Lead system design for Hadoop/Hive test environments, including YARN resource management, dynamic partitioning.
n
- Exposure to Zephyr-Jira-ServiceNow integrated test management systems with experience on API-driven automation.
n
- Design CI/CD test pipelines for PySpark/Hadoop jobs, incorporating artifact management, parallel execution, and blue-green deployments.
n
- Create data quality system designs using PySpark integrated with Hive metadata services.
n
- Design testing platforms, test data generators
n
- Mentor juniors on PySpark testing basics, contribute to testing strategy discussions
n
- Spark session configurations for memory and core allocations for both local and cluster manager settings
n
- Data handling with distributed file systems like HDFS and writing back to hive tables
n
- Implementation of Partitioning, caching techniques in organizing code for transformation pipelines
n
📌 Pyspark (Tamil Nadu)
🏢 Tata Consultancy Services
📍 Tamil Nadu
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.