Web Spiders is looking for a Senior Spark / PySpark Data Engineer with strong hands-on experience in Apache Spark, PySpark, Python, and large-scale data engineering. The ideal candidate will have strong experience designing and developing high-performance ETL/ELT pipelines and distributed data-processing solutions using Spark/PySpark, along with experience working with cloud-based data platforms and AWS services.

Location

Kolkata (Rajarhat–Newtown)
Type

Full Time
Department

Technology

Web Spiders is looking for a Senior Spark / PySpark Data Engineer with strong hands-on experience in Apache Spark, PySpark, Python, and large-scale data engineering.
The ideal candidate will have robust experience designing and developing high-performance ETL/ELT pipelines and distributed data-processing solutions using Spark/PySpark, along with experience working with cloud-based data platforms and AWS services.
If Spark + PySpark + Python is your core expertise and you enjoy solving complex data-processing and scalability challenges, we'd love to hear from you.




5+ Years Experience | Kolkata – Work from Office
Core Stack: Apache Spark • PySpark • Python • ETL/ELT • AWS • S3 • Glue • Redshift
* Immediate joiners preferred.*
Working Hours: Ability to work in the US Eastern Time Zone. Depending on project requirements, this may be adjusted to a half-day IST + half-day US EST schedule.
What You'll Do:
Design, develop, and optimize large-scale ETL/ELT pipelines using Apache Spark and PySpark.
Develop scalable data transformation and processing solutions using PySpark and Python.
Build distributed data-processing applications capable of handling large volumes of data.
Develop reusable and maintainable Spark/PySpark frameworks and data-processing components.
Optimize Spark jobs for performance, scalability, memory utilization, and execution efficiency.
Work with complex transformations, joins, aggregations, partitioning, and large datasets.
Implement data validation, quality checks, e

📌 Senior Spark (Kolkata)
🏢 Web Spiders
📍 Kolkata