Job DescriptionJob Title: PySpark Developer
nLocation: Chennai / Bangalore / Hyderabad / Pune
nNotice Period: Immediate to 30 Days
nJob Description :
nWe are seeking a skilled PySpark Developer with strong experience in Python, PySpark, SQL, and Data Warehousing concepts. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing pipelines and ETL solutions using Spark-based technologies.
nKey Responsibilities :
n
n
Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.n
Build and optimize Spark jobs for performance, reliability, and scalability.n
Process and transform large datasets using Spark SQL, DataFrames, and RDDs.n
Develop batch and real-time data processing solutions.n
Integrate data pipelines with Hive, HDFS, Snowflake, Redshift, and other data platforms.n
Collaborate with data engineers, analysts,



and business stakeholders.n
Monitor data pipelines, troubleshoot issues, and ensure SLA compliance.n
Follow coding best practices, version control, and CI/CD processes.n
Work with Hadoop ecosystem tools and cloud platforms such as AWS, Azure, or GCP.n

nRequired Skills:
n
n
Solid hands-on experience in Python and PySparkn
Expertise in Spark SQL, DataFrames, and RDDsn
Valuable knowledge of Hadoop (Hive, HDFS, YARN)n
Robust SQL and query optimization skillsn
Experience with Data Warehousing conceptsn
Knowledge of Parquet, Avro, JSON data formatsn
Experience with Git version controln
Familiarity with Airflow, Oozie, or similar scheduling toolsn
Exposure to AWS, Azure, or GCP is an added advantagen

📌 Pyspark Developer Python Sql Vasanthanagar
🏢 Tata Consultancy Services
📍 Vasanthanagar