Role & responsibilities
Design and develop scalable data pipelines using PySpark and Python.
Develop and optimize large-scale data processing workloads using Apache Spark.
Work extensively with AWS cloud services for data engineering solutions.
Build and maintain ETL/ELT pipelines for structured and semi-structured data.
Perform data transformation, cleansing, aggregation, and validation.
Optimize PySpark jobs for performance, scalability, and reliability.
Write complex and optimized SQL queries for data processing and analysis.
Troubleshoot production issues and improve pipeline performance.
Collaborate with data engineers, architects, analysts, and other stakeholders.
Follow coding standards and data-engineering best practices.

Preferred candidate profile
612 years of overall IT experience.
Solid hands-on experience in PySpark-based data engineering.
Robust practical knowledge of AWS is mandatory.
Good problem-solving and debugging skills.
Positive communication and stakeholder collaboration skills.

📌 Pyspark Developer With Aws Hyderabad (India)
🏢 Tata Consultancy Services
📍 India