Role- Big data engineer (Spark, Scala, AWS)

We are looking for a Data Engineer with strong experience in designing, developing, and maintaining scalable, secure, and high-performance ETL/data processing pipelines. The ideal candidate should have hands-on expertise in Apache Spark, Scala, AWS EMR, and Amazon S3, along with a solid understanding of big data technologies and cloud-based data platforms.

Key Responsibilities
Design, develop, and maintain scalable batch and big data processing pipelines using Apache Spark and Scala.
Build, deploy, and manage large-scale data processing solutions on AWS EMR.
Utilize Amazon S3 as the primary data lake and storage layer for ingesting, storing, and processing structured and unstructured data.
Optimize Spark applications for performance, scalability, reliability, and cost efficiency.
Develop reusable frameworks and components for data ingestion, transformation, validation, and processing.
Collaborate with data architects, business analysts, application teams, and stakeholders to understand data requirements and deliver robust data solutions.
Ensure data quality, integrity, security, and governance across all data pipelines.
Monitor, troubleshoot,



and resolve production issues related to data workflows and processing jobs.
Support deployment, release management, and ongoing maintenance of data platforms.
Follow best practices related to coding standards, testing, documentation, version control, and CI/CD processes.

Required Skills & Qualifications
Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline.
Strong hands-on experience in Scala programming.
Extensive experience with Apache Spark for large-scale distributed data processing.
Proven experience working with AWS EMR for big data workloads.
Solid knowledge of Amazon S3 and contemporary data lake architectures.
Solid understanding of ETL/ELT design principles and data pipeline development.
Strong SQL skills and experience working with large and complex datasets.
Good understanding of cloud-based architecture, performance tuning, job monitoring, and operational support.
Experience with version control systems such as Git/GitHub.
Excellent analytical, problem-solving, communication, and stakeholder management skills.

📌 Big Data Engineer Spark, Scala, Aws Hyderabad
🏢 Coforge
📍 Hyderabad