Job DescriptionJob Title: PySpark Developer
nLocation: Chennai / Bangalore / Hyderabad / Pune
nNotice Period: Immediate to 30 Days
nJob Description :
nWe are seeking a skilled PySpark Developer with strong experience in Python, PySpark, SQL, and Data Warehousing concepts. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing pipelines and ETL solutions using Spark-based technologies.
nKey Responsibilities :
n
- n
- Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.n
- Build and optimize Spark jobs for performance, reliability, and scalability.n
- Process and transform large datasets using Spark SQL, DataFrames, and RDDs.n
- Develop batch and real-time data processing solutions.n
- Integrate data pipelines with Hive, HDFS, Snowflake, Redshift, and other data platforms.n
- Collaborate with data engineers, analysts, and business stakeholders.n




- Monitor data pipelines, troubleshoot issues, and ensure SLA compliance.n
- Follow coding best practices, version control, and CI/CD processes.n
- Work with Hadoop ecosystem tools and cloud platforms such as AWS, Azure, or GCP.n

nRequired Skills:
n
- n
- Strong hands-on experience in Python and PySparkn
- Expertise in Spark SQL, DataFrames, and RDDsn
- Valuable knowledge of Hadoop (Hive, HDFS, YARN)n
- Strong SQL and query optimization skillsn
- Experience with Data Warehousing conceptsn
- Knowledge of Parquet, Avro, JSON data formatsn
- Experience with Git version controln
- Familiarity with Airflow, Oozie, or similar scheduling toolsn
- Exposure to AWS, Azure, or GCP is an added advantagen

📌 PySpark Developer | Python | SQL (Vasanthanagar)
🏢 Tata Consultancy Services
📍 Vasanthanagar