We are looking for a Technical Lead – PySpark with solid hands-on experience in building and optimizing scalable data processing solutions. The ideal candidate should have solid expertise in PySpark, Python, SQL, and data engineering , along with the ability to provide technical guidance to the development team.
Key Responsibilities
- Design, develop, and maintain scalable data processing solutions using PySpark .
- Build and optimize ETL/data pipelines for large datasets.
- Perform performance tuning and optimization of Spark/PySpark jobs.
- Troubleshoot complex data processing and application issues.
- Conduct code reviews and ensure development best practices.
- Provide technical guidance and mentoring to team members.
- Collaborate with cross-functional teams to translate requirements into technical solutions.
- Ensure solutions are scalable, reliable, and maintainable.
Required Skills
- Strong hands-on development experience in PySpark .
- Strong programming skills in Python .
- Good understanding of Apache Spark and distributed data processing.
- Strong knowledge of SQL and data transformations.
- Hands-on experience developing ETL/data pipelines .
- Experience in Spark/PySpark performance optimization.
- Good problem-solving and troubleshooting skills.
- Experience in technical leadership, code reviews, and team guidance.
Preferred Skills
- Exposure to AWS, Azure, or GCP data platforms.
- Understanding of big-data and modern data-engineering ecosystems.
- Familiarity with Git, CI/CD, and software development best practices.
📌 Technical Lead – PySpark / Pune /
🏢 Tata Consultancy Services
📍 Pune
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.