PySpark Architect Location Bangalore Pune Kolkata Experience Required 7 to 13 Years Role Overview We are seeking a highly skilled and motivated PySpark Manager to lead our data engineering initiatives The ideal candidate will have deep expertise in PySpark Hadoop AWS Teradata Scala and data visualization tools You will be responsible for designing building and managing large-scale data processing solutions while leading a team of data engineers to deliver high-quality outcomes Key Responsibilities 1 Leadership Team Management Lead and mentor a team of data engineers ensuring high performance and skilled growth Oversee project timelines quality and deliverables for multiple data initiatives 2 Data Engineering Development Design and implement robust data pipelines and workflows using PySpark and Hadoop Optimize performance and scalability of big data solutions for complex data sets Develop and integrate solutions with AWS services for cloud-based data processing 3 Technical Expertise Leverage expertise in PySpark Hadoop and Scala to solve complex technical challenges Work with Teradata for large-scale data warehousing solutions Utilize data visualization tools e g Tableau Power BI to provide insights and reporting solutions 4 Stakeholder Collaboration Collaborate with cross-functional teams including business stakeholders and product managers to understand requirements and deliver solutions Provide thought leadership on best practices in big data technologies and architecture Required Skills Qualifications Technical Expertise Proficiency in PySpark Hadoop AWS S3 Glue EMR Redshift and Teradata Hands-on experience with Scala for data engineering tasks Familiarity with data visualization tools like Tableau Power BI or similar platforms Experience 7-13 years of experience in data engineering with a focus on big data technologies Proven track record of managing end-to-end data engineering projects and leading teams Educational Background Bachelor s or Master s degree in Computer Science Information Technology or a related field Soft Skills Strong analytical and problem-solving skills Excellent communication and stakeholder management abilities Ability to work in a fast-paced dynamic environment Preferred Skills Certification in AWS or related big data tools Experience in real-time data streaming platforms like Kafka Knowledge of DevOps practices including CI CD pipelines and infrastructure as code Requirements Talent Worx is a growing services recruitment consulting firm we are hiring for our client which is a Global and leading provider of financial intelligence data analytics and AI-driven solutions empowering businesses worldwide with insights for confident decision-making Join to work on cutting-edge technologies drive digital transformation and shape the future of global markets

📌 Pyspark Architect (Pune)
🏢 Talent Worx
📍 Pune