15 Nov
|
Infosys
|
Bengaluru
Job highlights
- 3+ years experience in AWS Glue, PySpark, SQL, and data lake technologies with a B.E./B.Tech degree
- Design, develop, and optimize scalable ETL/ELT pipelines using AWS Glue and related AWS services
We are looking for an experienced AWS Glue Developer / Data Engineer to design, develop, and optimize scalable ETL/ELT data pipelines on AWS. The candidate should have strong hands-on experience with AWS Glue,Python, PySpark, SQL, data lake technologies, and AWS data services.
Roles Responsibilities sponsibilities
- Design, develop, and maintain ETL/ELT pipelines using AWS Glue.
- Develop AWS Glue jobs using PySpark, and SQL for large-scale data processing.
- Build data ingestion and transformation pipelines across structured and semi-structured data sources.
- Configure and manage AWS Glue Data Catalog, Jobs, Workflows, and Triggers.
- Work with AWS services such as Amazon S3, Redshift, and IAM.
- Implement data transformations, validations, data-quality checks, and reconciliation processes.
- Optimize Glue/Spark jobs for performance, scalability, reliability, and cost efficiency.
- Implement incremental loads, partitioning strategies, error handling, restart/recovery, and audit frameworks.
- Troubleshoot ETL pipeline failures and production performance issues.
- Implement appropriate security controls including IAM roles, encryption, and access management.
- Support migration of legacy/mainframe or on-premises ETL workloads to the AWS cloud data platform.
- Develop automated deployment processes using CI/CD and Infrastructure-as-Code practices where applicable.
- Collaborate with data architects, analysts, business teams, and cloud/platform teams to translate requirements into scalable data solutions.
- Prepare technical design documents, mapping specifications, operational documentation, and deployment/runbook documentation.
Additional Responsibility
Required Technical Skills
- AWS Glue
- PySpark / Apache Spark
- SQL
- Amazon S3
- Data Lake / Cloud Data Warehouse concepts
- ETL/ELT design and development
- Performance optimization and troubleshooting
Experience
- 3+ years of overall Data Engineering / ETL development experience.
- Solid hands-on experience developing production-grade AWS Glue,Python and PySpark solutions.
- Experience handling high-volume datasets and complex transformations in AWS environments.
Preferred Certification
- AWS Certified Data Engineer - Associate or other relevant AWS certification.
Educational Requirement
Bachelor of Engineering,Bachelor Of Technology
Preferred Skills
Technology->Cloud Platform->AWS Database,Technology->Data On Cloud - Platform->AWS
Service Line
Data Analytics Unit
📌 Aws Developer (Bengaluru)
🏢 Infosys
📍 Bengaluru