21 Sep
|
Weekday AI
|
Mumbai
Description
???? ???? ?? ??? ??? ?? ??? ???????'? ???????
?????? ?????: ?? ??????? - ?? ??????? (?? ??? ??-?? ???)
Experience: 3+ yrs
Location: Mumbai, Maharashtra, India
Job Type: Full-time
We are looking for an experienced Data Engineer to design, develop, and maintain scalable data platforms and pipelines using AWS, Apache Spark, Python, SQL, Kafka, and modern data engineering frameworks.
The role will focus on building reliable data solutions for high-volume batch and real-time workloads, including data ingestion, transformation, processing, orchestration, storage, and delivery. The ideal candidate will have strong hands-on experience with AWS data services and distributed data processing, along with a solid understanding of data modelling, streaming architectures, data quality, and pipeline optimisation.
Requirements
KEY RESPONSIBILITIES
- Design, develop, and maintain end-to-end data pipelines for high-volume data ingestion, transformation, processing, and delivery.
- Build scalable Spark-based ETL/ELT workflows for both batch and real-time data processing.
- Develop data ingestion solutions using Kafka, Amazon Kinesis, and other streaming technologies.
- Build and manage data lakes, warehouses, and lakehouse solutions using AWS S3, Glue, Redshift, Athena, and EMR.
- Develop productive data models using dimensional modelling, star schemas, partitioning, and other data engineering practices.
- Implement data quality checks, validation rules, monitoring, and error-handling mechanisms.
- Develop automated workflows using Airflow, MWAA, AWS Step Functions, or similar orchestration tools.
- Collaborate with Data Analysts and Data Scientists to deliver clean, structured, and analytics-ready datasets.
- Optimise data pipelines for performance, scalability, reliability, and AWS cost efficiency.
- Integrate data from multiple internal and external systems while maintaining data consistency and reliability.
- Develop Python-based automation and data-processing solutio
📌 Data Engineer (Mumbai)
🏢 Weekday AI
📍 Mumbai