PySpark Developer Job DescriptionAbout The Company

Tata Consultancy Services (TCS) is a global leader in IT services, consulting, and business solutions. With a rich history spanning several decades, TCS has established itself as a trusted partner for organizations across diverse industries. The company is committed to delivering innovative technology solutions that drive business growth and digital transformation.

TCS’s extensive portfolio includes cloud computing, data analytics, artificial intelligence, and enterprise solutions, supported by a vast global delivery network. Employees at TCS benefit from a collaborative work environment, continuous learning opportunities, and a culture that values diversity and inclusion. As part of its commitment to excellence and innovation, TCS continually invests in cutting-edge technologies and talent development to stay ahead in the rapidly evolving digital landscape.

About The Role

We are currently seeking a highly skilled PySpark Developer to join our dynamic team. The ideal candidate will play a crucial role in designing, developing, and maintaining large-scale data processing pipelines and ETL solutions using Spark-based technologies. This role requires a deep understanding of Python, PySpark, SQL, and Data Warehousing concepts to enable efficient data transformation and integration.

The successful candidate will collaborate with cross-functional teams including data engineers, analysts, and business stakeholders to deliver scalable and reliable data solutions that support strategic decision-making. This position offers an excellent opportunity to work with cutting-edge big data technologies within a globally recognized organization dedicated to innovation and excellence.

Qualifications The ideal candidate should possess a strong educational background in Computer Science, Information Technology, or related fields. Hands-on experience with Python and PySpark is essential,



along with expertise in Spark SQL, DataFrames, and RDDs. Candidates should have a positive understanding of Hadoop ecosystem components such as Hive, HDFS, and YARN. Proficiency in SQL and query optimization techniques is required.

Experience working with Data Warehousing concepts and familiarity with data formats like Parquet, Avro, and JSON is highly desirable. Knowledge of version control systems such as Git, along with experience using scheduling tools like Airflow or Oozie, will be advantageous. Exposure to cloud platforms like AWS, Azure, or GCP will be considered a plus.

Strong problem-solving skills, attention to detail, and the ability to work effectively in a team setting are essential for success in this role.

Responsibilities The PySpark Developer will be responsible for designing, developing, and maintaining scalable ETL/ELT pipelines utilizing PySpark. This includes building optimized Spark jobs to ensure high performance, reliability, and scalability of data processing tasks. The role involves processing and transforming large datasets using Spark SQL, DataFrames, and RDDs, as well as developing batch and real-time data processing solutions to meet business needs.

The developer will integrate data pipelines with various data platforms such as Hive, HDFS, Snowflake, and Redshift, ensuring seamless data flow across systems. Collaboration with data engineers, analysts, and business stakeholders is vital to understand requirements and deliver effective solutions. Monitoring data pipelines, troubleshooting issues,



and ensuring adherence to SLAs are key responsibilities.

The developer will also follow best coding practices, utilize version control, and participate in CI/CD processes to ensure code quality and deployment efficiency. Working with Hadoop ecosystem tools and cloud platforms such as AWS, Azure, or GCP will be part of the responsibilities to support scalable and flexible data architecture.

Benefits

Joining Tata Consultancy Services offers numerous benefits, including competitive compensation packages, comprehensive health insurance, and retirement plans. Employees have access to ongoing training and professional development programs to enhance their skills and career growth. TCS promotes a healthy work-life balance through versatile working arrangements and wellness initiatives.

The organization fosters a cooperative and inclusive work environment, encouraging innovation and continuous learning. Additionally, employees benefit from exposure to cutting-edge technologies and diverse projects across industries, providing valuable experience and opportunities for advancement within a global organization. TCS also offers various employee engagement activities, recognition programs, and support systems to ensure a positive and motivating workplace experience.

Equal Opportunity

Tata Consultancy Services is an equal opportunity employer. We are committed to creating an inclusive setting where all employees and applicants are treated with respect and dignity. We do not discriminate based on race, religion, gender, age, sexual orientation, disability, or any other protected characteristic.

Our hiring practices are designed to promote diversity and ensure equal access to employment opportunities. We believe that a diverse workforce enhances our ability to innovate and serve our clients effectively, and we are dedicated to fostering a culture of inclusion and belonging for all employees.

📌 Python Developer (India)
🏢 FetchJobs.co
📍 India