We are looking for a skilled Data Engineer with expertise in PySpark, AWS, and SQL to support data processing and analytical initiatives. This role involves working closely with data engineering and data science teams to build, maintain, and optimize large-scale data pipelines and integrations on AWS. The ideal candidate will be proficient in ETL processes using PySpark and SQL, with a deep understanding of cloud data infrastructure, specifically within the AWS ecosystem.

Key Responsibilities :

Data Pipeline Development :
- Design, build, and optimize ETL pipelines using PySpark for data ingestion, transformation, and storage on AWS.
- Collaborate with stakeholders to understand data requirements, translating them into scalable data solutions.

Cloud Infrastructure Management :
- Develop and manage AWS services such as S3, Glue, Lambda, EMR, Redshift, and RDS for data processing and storage.
- Implement data workflows that handle both batch and real-time processing needs, ensuring low-latency and efficient data access.

Database Management :




- Write and optimize complex SQL queries for data extraction and transformation from AWS RDS, Redshift, and other SQL-based databases.
- Leverage indexing, partitioning, and caching techniques to improve query performance on large datasets.

Data Quality and Governance :
- Ensure data accuracy, completeness, and consistency throughout the data lifecycle.
- Implement best practices for data quality, governance, and security using AWS-native tools and third-party solutions.

Automation and Optimization :
- Automate repetitive tasks and optimize workflows, ensuring productive and resilient data processing.
- Use CI/CD pipelines and version control (e.g., Git) to deploy and manage data workflows.

Troubleshooting and Support :
- Identify and resolve issues within the data pipeline, providing solutions for data access and query performance.
- Support the team in data analytics and data science initiatives by preparing data in an acc

📌 Data Engineer (Noida)
🏢 Forward Eye Technologies
📍 Noida