Roles and Responsibilities
Design, develop, and maintain ETL/ELT pipelines using Python and AWS Glue.
Extract, transform, and load data from multiple sources into target systems.
Develop data processing and transformation jobs using Python and PySpark.
Use Amazon S3 for data storage, organization, and processing of structured and semi-structured data.
Implement data cleansing, validation, transformation, and deduplication logic.
Work with AWS Glue Data Catalog and crawlers to manage metadata.
Write and optimize SQL queries for data extraction, transformation, and validation.
Monitor ETL jobs, troubleshoot failures, and improve pipeline performance.
Implement error handling, logging, and data quality checks.
Collaborate with data engineers, analysts, and other stakeholders to understand data requirements.




Maintain technical documentation and follow data engineering best practices.

Must-Have Skills
Robust hands-on experience in ETL development and data pipeline implementation.
Proficiency in Python programming and scripting.
Mandatory experience with AWS Glue and Amazon S3.
Experience with PySpark for large-scale data processing and transformations.
Solid SQL skills, including joins, subqueries, aggregations, and query optimization.
Experience working with CSV, JSON, and Parquet data formats.
Understanding of data warehousing concepts and data quality practices.
Experience in debugging, troubleshooting, and optimizing ETL pipelines.

📌 Aws Data Engineer Hyderabad
🏢 CGI
📍 Hyderabad