Job Description

Job Title: PySpark Developer

n

Location: Chennai / Bangalore / Hyderabad / Pune

n

Notice Period: Immediate to 30 Days

n

Job Description :

n

We are seeking a skilled PySpark Developer with robust experience in Python, PySpark, SQL, and Data Warehousing concepts. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing pipelines and ETL solutions using Spark-based technologies.

n

Key Responsibilities :

n

n
Design, develop, and maintain scalable ETL/ELT pipelines using PySpark.
n
Build and optimize Spark jobs for performance, reliability, and scalability.
n
Process and transform large datasets using Spark SQL, DataFrames, and RDDs.
n
Develop batch and real-time data processing solutions.
n
Integrate data pipelines with Hive, HDFS, Snowflake, Redshift, and other data platforms.
n
Collaborate with data engineers, analysts,



and business stakeholders.
n
Monitor data pipelines, troubleshoot issues, and ensure SLA compliance.
n
Follow coding best practices, version control, and CI/CD processes.
n
Work with Hadoop ecosystem tools and cloud platforms such as AWS, Azure, or GCP.
n

n

Required Skills:

n

n
Robust hands-on experience in Python and PySpark
n
Expertise in Spark SQL, DataFrames, and RDDs
n
Positive knowledge of Hadoop (Hive, HDFS, YARN)
n
Strong SQL and query optimization skills
n
Experience with Data Warehousing concepts
n
Knowledge of Parquet, Avro, JSON data formats
n
Experience with Git version control
n
Familiarity with Airflow, Oozie, or similar scheduling tools
n
Exposure to AWS, Azure, or GCP is an added advantage
n

📌 Pyspark Developer Python Sql Bengaluru (India)
🏢 Tata Consultancy Services
📍 India