Job DescriptionJob Title: PySpark Developer

NLocation: Chennai / Bangalore / Hyderabad / Pune

NNotice Period: Immediate to 30 Days

NJob Description:

NWe are seeking a skilled PySpark Developer with solid experience in Python, PySpark, SQL, and Data Warehousing concepts. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing pipelines and ETL solutions using Spark-based technologies.

NKey Responsibilities:

N

n
Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.
N
Build and optimize Spark jobs for performance, reliability, and scalability.
N
Process and transform large datasets using Spark SQL, DataFrames, and RDDs.
N
Develop batch and real-time data processing solutions.
N
Integrate data pipelines with Hive, HDFS, Snowflake, Redshift, and other data platforms.
N
Collaborate with data engineers, analysts, and business stakeholders.
N




Monitor data pipelines,troubleshoot issues, and ensure SLA compliance.
N
Follow coding best practices, version control, and CI/CD processes.
N
Work with Hadoop ecosystem tools and cloud platforms such as AWS, Azure, or GCP.
N

nRequired Skills:

N

n
Robust hands-on experience in Python and PySpark
N
Expertise in Spark SQL,DataFrames, and RDDs
N
Valuable knowledge of Hadoop (Hive, HDFS, YARN)
N
Strong SQL and query optimization skills
N
Experience with Data Warehousing concepts
N
Knowledge of Parquet, Avro, JSON data formats
N
Experience with Git version control
N
Familiarity with Airflow, Oozie, or similar scheduling tools
N
Exposure to AWS, Azure,or GCP is an added advantage
N

📌 Hiring: Pyspark Developer Python Sql Bengaluru
🏢 Tata Consultancy Services
📍 Bengaluru