Experience: 7–8 Years
Employment Type: Full-Time
Work Mode: Remote

Job Summary

We are looking for an experienced Big Data Engineer with 7–8 years of experience in designing, developing, and maintaining scalable big data pipelines and data processing solutions.

The ideal candidate should have solid hands-on experience with Scala, PySpark, AWS Glue, big data pipeline development, and data validation/profiling, along with good knowledge of APIs, databases, testing, and modern web technologies.

Key Responsibilities
- Design, develop, and maintain scalable big data pipelines.
- Develop data processing solutions using Scala and PySpark.
- Build and manage ETL/ELT pipelines using AWS Glue.
- Perform data validation, profiling, quality checks, and transformation.
- Work with large and complex datasets to ensure data accuracy and consistency.
- Develop and integrate REST APIs where required.
- Work with SQL and NoSQL databases including PostgreSQL, MongoDB, and Elasticsearch.
- Implement testing and validation using Scala/Java.




- Troubleshoot pipeline and data-related issues and optimize performance.
- Collaborate with data engineers, developers, QA, and business teams.
- Follow best practices for data engineering, testing, documentation, and deployment.

Primary Skills
- Scala
- PySpark
- AWS Glue
- Big Data Pipeline Development
- Data Validation & Profiling

Secondary Skills
- REST APIs
- Async JavaScript
- SQL / NoSQL
- PostgreSQL
- MongoDB
- Elasticsearch
- Web Design
- Testing with Scala / Java

Required Qualifications
- 7–8 years of experience in Big Data/Data Engineering.
- Strong hands-on experience with Scala and PySpark.
- Experience developing and maintaining big data pipelines.
- Strong experience with AWS Glue.
- Experience in data validation, profiling, and data quality.
- Good knowledge of SQL/NoSQL databases.
- Solid debugging and problem-solving skills.
- Good communication and collaboration skills.

Work Location: Remote

📌 Big Data Engineer (India)
🏢 WebSenor InfoTech
📍 India