PySpark Developer Job DescriptionAbout The Company

Tata Consultancy Services (TCS) is a global leader in IT services, consulting, and business solutions. With a rich history spanning over several decades, TCS has established itself as a trusted partner for organizations worldwide, delivering innovative technology solutions that drive business transformation. The company’s commitment to excellence, sustainability, and customer-centricity has earned it a reputation for delivering high-quality services across various industries, including banking and finance, healthcare, manufacturing, retail, and more.

TCS prides itself on fostering a diverse and inclusive work environment, investing in employee development, and leveraging cutting-edge technology to create impactful solutions for its clients.

About The Role

We are seeking a highly skilled PySpark Developer to join our dynamic team at Tata Consultancy Services. In this role, you will be responsible for designing, developing, and maintaining large-scale data processing pipelines and ETL solutions that support our clients’ data-driven decision-making processes. The ideal candidate will have a strong background in Python, PySpark, SQL, and Data Warehousing concepts, with hands-on experience in building scalable and efficient data processing systems.

You will work closely with data engineers, analysts, and business stakeholders to ensure data pipelines are optimized for performance, reliability, and scalability. This position offers an exciting opportunity to work on innovative projects leveraging Spark-based technologies across cloud and on-premise environments, contributing significantly to the organization’s data ecosystem.





Qualifications The ideal candidate should possess a comprehensive set of technical skills and relevant experience, including:

- Proven hands-on experience in Python and PySpark development
- Expertise in Spark SQL, DataFrames, and RDDs for large-scale data processing
- Robust knowledge of Hadoop ecosystem components such as Hive, HDFS, and YARN
- Excellent SQL skills with a focus on query optimization and performance tuning
- Understanding of Data Warehousing concepts and architecture
- Experience working with data formats such as Parquet, Avro, and JSON
- Proficiency with version control systems, especially Git
- Familiarity with scheduling and workflow orchestration tools like Airflow or Oozie
- Knowledge of cloud platforms such as AWS, Azure, or GCP is advantageous
- Strong problem-solving skills and the ability to work collaboratively in a team environment

Responsibilities The PySpark Developer will be responsible for a variety of tasks, including:

- Designing, developing, and maintaining scalable ETL/ELT pipelines utilizing PySpark
- Building and optimizing Spark jobs to ensure high performance, reliability, and scalability
- Processing and transforming large datasets using Spark SQL, DataFrames, and RDDs
- Developing both batch and real-time data processing solutions to meet business needs




- Integrating data pipelines with platforms such as Hive, HDFS, Snowflake, Redshift, and others
- Collaborating with cross-functional teams including data engineers, analysts, and business stakeholders to gather requirements and deliver solutions
- Monitoring data pipelines for performance issues, troubleshooting errors, and ensuring adherence to SLAs
- Following best coding practices, version control standards, and continuous integration/continuous deployment (CI/CD) processes
- Working with Hadoop ecosystem tools and cloud services to enhance data processing capabilities

Benefits Joining Tata Consultancy Services as a PySpark Developer offers numerous benefits, including:

- Competitive salary packages aligned with industry standards
- Opportunities for qualified growth and career advancement within a global organization
- Access to extensive training programs and certifications to enhance your technical skills
- Exposure to cutting-edge technologies and innovative projects across diverse industries
- Comprehensive health and wellness benefits for you and your family
- Versatile work arrangements and a supportive work environment
- Employee engagement activities and a collaborative culture that values diversity and inclusion

Equal Opportunity Tata Consultancy Services is an equal opportunity employer. We are committed to creating an inclusive environment for all employees and applicants, regardless of race, gender, age, religion, national origin, disability, or any other protected characteristic. We believe in fostering a workplace where everyone can thrive and contribute to our collective success.

📌 Python Developer (India)
🏢 FetchJobs.co
📍 India