About The Role

We are looking for an experienced AWS Data Engineer with strong expertise in PySpark, SQL, and AWS Redshift to design, develop, and optimize scalable data pipelines and cloud-based data platforms. The ideal candidate will have hands-on experience with AWS data services, large-scale data processing, ETL/ELT development, and data warehousing.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using PySpark and AWS services.
- Build and optimize ETL/ELT workflows for structured and unstructured data.
- Develop and manage enterprise data warehouses using AWS Redshift.
- Write complex, optimized SQL queries for data extraction, transformation, and analysis.
- Ingest data from APIs, databases, flat files, and streaming platforms.
- Perform data cleansing, transformation, validation, and quality checks.
- Optimize Redshift schemas, tables, and queries for performance and scalability.
- Implement data governance, security, and compliance best practices.
- Monitor, troubleshoot, and resolve data pipeline and performance issues.
- Collaborate with cross-functional teams and support production deployments.

Required Technical Skills

- Solid hands-on experience with PySpark and Apache Spark.




- Advanced SQL development and performance optimization.
- Amazon Redshift – Mandatory.
- AWS S3 and AWS Glue.
- AWS Lambda, IAM, CloudWatch, Step Functions, and Secrets Manager.
- ETL/ELT pipeline development and data warehousing.
- Data partitioning, performance tuning, and data quality validation.
- Python and shell scripting.
- Git and CI/CD pipelines.

Preferred Skills

- Data Lake and Lakehouse architectures.
- Databricks or Amazon EMR.
- Apache Airflow or AWS MWAA.
- Kafka or real-time streaming platforms.
- Terraform, CloudFormation, or Infrastructure as Code.
- Snowflake and data governance tools.

Qualifications

- Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
- 5+ years of experience in Data Engineering.
- Proven experience developing large-scale data pipelines and cloud data warehouse solutions.
- Strong analytical, troubleshooting, and communication skills.
- Willingness to work onsite in Chennai.

5 Primary Skills

- PySpark / Apache Spark
- AWS Redshift – Mandatory
- Advanced SQL
- AWS Glue and S3
- ETL/ELT Pipeline Development

Skills: aws glue,pyspark,aws,advanced sql,s3,elt,apache,etl

📌 AWS Data Engineer (Chennai)
🏢 Durapid Technologies Private
📍 Chennai