Job Summary: An experienced PySpark Developer / Leads to design, develop, and maintain large-scale data processing solutions using Apache Spark and Python. The ideal candidate should have a robust background in data engineering, data processing, and cloud-based data platforms, big data ecosystems and performance optimization

Responsibilities:
Data Processing: Design, develop, and maintain data processing solutions using PySpark, Apache Spark, and Python.
Data Pipeline Development: Develop and optimize data pipelines using PySpark, Apache Spark, and cloud-based data platforms.
Data Integration: Integrate data from various sources, including relational databases, NoSQL databases, and cloud storage.
Data Transformation: Develop and implement data transformation logic using PySpark, Apache Spark, and Python.
Collaboration: Work with cross-functional teams to identify and prioritize project requirements,



provide technical guidance, and ensure data quality.

Required Skills:
PySpark: In-depth knowledge of PySpark, Apache Spark, and Python.
Data Processing: Solid understanding of data processing concepts, including data ingestion, data transformation, and data storage.
Cloud Experience: Experience with cloud-based data platforms, including AWS, Azure, or Google Cloud.
Expertise in DataFrames & Spark SQL, Spark Streaming with Apache Kafka real-time data ingestion pipelines
Very positive conceptual understanding of Multithreading, distributed computing concepts of Pyspark
Programming: Proficiency in programming languages, including Python, Java, or Scala.
Communication: Excellent communication and collaboration skills.

📌 Walk In Pyspark Developer Chennai
🏢 Tata Consultancy Services
📍 Chennai