We are seeking an experienced Scala / Pyspark associate to design, develop, and maintain scalable data solutions on the Amazon Cloud Platform (AWS). The ideal candidate will have strong expertise in building up-to-date data pipelines, data warehousing, big data processing, and cloud-native analytics solutions. The role requires close collaboration with business stakeholders, data architects, data scientists, and application teams to deliver reliable and high performance data platforms. Key Responsibilities* - Design, develop, and optimize scalable data pipelines using scala and Pyspark in AWS environment. - Build and maintain batch and real-time data ingestion and processing frameworks. - Develop enterprise-grade data warehousing solutions using Scala and Pyspark. - Implement ETL/ELT processes for structured and unstructured data. - Analyze existing Scala and Spark applications and identify migration requirements. - Convert Scala-based ETL, batch, and streaming pipelines into PySpark frameworks. - Optimize PySpark jobs for performance, scalability, and resource utilization. - Support cloud modernization initiatives on AWS/Databricks/Snowflake platforms. berribot jd version 3 berribot jd version 3 - Integrate data from multiple sources including databases, APIs, files, and streaming platforms. - Ensure data quality, governance, security, and compliance across data platforms.



- Automate deployment and operational processes using CI/CD and Infrastructure as Code (IaC). - Monitor data pipelines and troubleshoot production issues. - Collaborate with Data Architects, Business Analysts, and Data Scientists to translate business requirements into technical solutions. - Implement data models, metadata management, and data lineage best practices. - Support migration of on-premises or multi-cloud data platforms to AWS. - Lead the migration, modernization, and optimization of Scala and Apache Spark workloads on Cloud environment, including conversion of Scala-based Spark applications to PySpark, performance tuning, cluster optimization, dependency management, and ensuring scalable, cost-effective, and resilient data processing solutions. Required Technical Skills - Scala - PySpark - AWS Glue - AWS S3 - Step Functions - Glue - Lambda - Pub/Sub - Python - SQL (Advanced) - Event Bridge - ECS - EKS berribot jd version 3 - Snowflake Section IV - Job Qualifications & Skills Section Details / Example Content Domain CMTS Soft Skills - Excellent communication - Team collaboration - Documentation and knowledge sharing Education Requirements Bachelor's/master’s in computer science or equivalent (Preferred) Certifications AWS Cloud Data Engineer (Preferred) Snow pro certifications Section V - Sample Questions for Training Model

📌 Hiring: Software Associate (Hyderabad)
🏢 Tata Consultancy Services
📍 Hyderabad