Job Description PySpark Data Engineer

Job Title: PySpark Data Engineer
Experience: 5-10 Years
Location: Chennai

Job Summary

We are looking for a skilled PySpark Data Engineer to design, develop, and optimize large-scale data processing pipelines. The ideal candidate should have solid expertise in PySpark, Spark ecosystem, SQL, ETL development, and cloud/big data technologies.

Key Responsibilities

- Develop and optimize data pipelines using PySpark and Apache Spark.
- Perform data ingestion, transformation, and processing of large datasets.
- Design scalable ETL/ELT solutions for data warehousing and analytics.
- Work with distributed computing frameworks and big data technologies.
- Integrate data from multiple sources and ensure data quality.
- Collaborate with Data Scientists, Analysts,



and Business teams.
- Troubleshoot performance issues and optimize Spark jobs.
- Follow best practices for data governance, security, and code quality.

Required Skills

- Strong hands-on experience in PySpark development.
- Expertise in Spark SQL, DataFrames, and Spark optimization techniques.
- Experience with Hadoop ecosystem (Hive, HDFS, Kafka).
- Good knowledge of SQL and database concepts.
- Experience in ETL pipeline development.
- Knowledge of Azure Databricks, Azure Data Factory, or AWS EMR is preferred.
- Experience with Git, CI/CD, and Agile methodologies.

📌 Tcs Hiring For Pyspark Data Engineer (Chennai)
🏢 Tata Consultancy Services
📍 Chennai