Are you passionate about building scalable big data solutions and high-performance data pipelines? Join our team and work with cutting-edge technologies to develop enterprise-grade data platforms.
Experience Required
- 4+ years of software engineering experience developing Big Data pipelines.
- 3+ years of hands-on experience programming in Scala and PySpark.
- 3+ years of experience building solutions on AWS, including AWS Glue.
- 3+ years of experience working with Big Data file formats, including Parquet.
- 1+ years of experience with Data Validation, Data Profiling, or building Rule-Based Data Engines.
- 1+ years of experience leveraging AI coding assistants or code-generation tools (e.g., GitHub Copilot, Codex) in daily software development tasks.
Key Responsibilities
- Design, build, and optimize a highly scalable, rule-based data validation engine.
- Develop robust, high-performance Big Data pipelines using Scala, PySpark,
and AWS Glue.
- Write clean, well-tested code utilizing optimized formats like Parquet for effective storage and retrieval.
- Perform detailed data validation and profiling to ensure data quality and consistency across data systems.
- Use enterprise-approved AI tools to streamline software development workflows and automate repetitive tasks.
- Collaborate with cross-functional teams to translate business requirements into scalable technical solutions.
- Evaluate emerging AI and Big Data technologies to drive innovation.
- Participate in code reviews, testing, debugging, and performance optimization.
Required Skills
- Scala
- PySpark
- AWS
- AWS Glue
- Parquet
- Big Data Pipelines
- Data Validation
- Data Profiling
- AI Coding Assistants (GitHub Copilot/Codex)
📌 Big Data Engineer (India)
🏢 WebSenor
📍 India
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.