Job Details:-

Location: Gurugram (Work From Office)

Experience: 6+ Years

Job Summary

We are seeking a highly skilled Data Engineer with 6+ years of experience and solid hands-on expertise in PySpark, AWS, and Python to build and maintain scalable, high-performance data pipelines. The role involves working on large-scale distributed data processing systems and delivering reliable data solutions to support analytics and business intelligence.

Key Responsibilities

- Design, develop, and optimize ETL/ELT pipelines using Python and PySpark
- Build and maintain large-scale data processing frameworks on AWS
- Develop batch and near-real-time data pipelines
- Process structured and semi-structured data using Apache Spark
- Ingest data from multiple sources including APIs, databases, and streaming platforms
- Optimize Spark jobs for performance and cost efficiency
- Build and manage data lakes using AWS S3
- Implement data workflows using orchestration tools (Airflow, AWS Glue, Step Functions)
- Ensure data quality, validation, and reliability
- Monitor production pipelines and resolve failures
- Collaborate with data scientists, analysts, and business stakeholders




- Follow best practices for security, governance, and compliance on AWS

Required Skills & Qualifications
- 6+ years of experience as a Data Engineer
- Strong proficiency in Python and PySpark
- Hands-on experience with AWS services, including:
- S3, EC2, EMR, Glue, Lambda
- Redshift, Athena
- CloudWatch, IAM
- Strong experience with Apache Spark and distributed data processing
- Advanced SQL skills for querying and data transformation
- Experience with ETL tools and frameworks
- Experience with data warehousing and data modeling
- Strong understanding of data partitioning, performance tuning, and cost optimization
- Experience with Git and CI/CD pipelines

Good to Have / Preferred
- Experience with streaming technologies (Kafka, Kinesis)
- Knowledge of Delta Lake / Iceberg / Hudi
- Experience with Terraform or CloudFormation
- Exposure to BI tools (Power BI, Tableau, QuickSight)
- Experience supporting production data systems in enterprise environmentsRole & responsibilities

📌 Data Engineer (Ahmedabad)
🏢 Walking Tree
📍 Ahmedabad