Key Responsibilities:

- Design and develop scalable data processing applications using Apache Spark with Scala
- Build and optimize batch and streaming data pipelines
- Process large datasets from multiple data sources with high reliability and performance
- Write clean, productive, and reusable Scala code
- Tune Spark jobs for performance, memory, and resource optimization
- Collaborate with data engineers, analysts, and stakeholders to understand data requirements
- Debug and resolve production issues related to data pipelines
- Participate in code reviews and follow best practices for software development
- Maintain documentation for data workflows and solutions

Required Skills:

- Strong hands-on experience with Apache Spark and Scala
- Good understanding of RDDs, DataFrames, and Datasets
- Experience with SQL / Spark SQL
- Hands-on experience with big data storage systems such as HDFS, Hive, or cloud data lakes
- Knowledge of Linux/Unix environments
- Experience with version control tools like Git
- Understanding of data structures, algorithms, and distributed computing concepts

📌 Spark Scala Developer - S (Chennai)
🏢 Tata Consultancy Services
📍 Chennai