Apply on Kit Job: kitjob.in/job/42qevu

Data Engineer – AWS Data Platform

n

Job Summary

n

We are looking for a highly skilled and motivated Data Engineer with strong expertise in AWS data services to join our data platform team. The ideal candidate will have hands-on experience designing scalable data pipelines, workflow orchestration frameworks, and large-scale data migration solutions.

n

This role will be responsible for building robust cloud-native data engineering solutions on AWS, migrating datasets from legacy systems and data warehouses, and ensuring secure and efficient data processing pipelines across distributed environments.

n

Key Responsibilities

n

AWS Data Pipeline Development

n

Design and implement scalable ETL/ELT data pipelines using AWS Glue, AWS Lambda, and AWS S3.

n

Build and maintain high-performance data ingestion frameworks for processing large-scale datasets.

n

Implement data pipelines for data warehousing and analytics platforms such as AWS Redshift.

n

Optimize storage and querying strategies using AWS S3 data lakes.

n

Data Workflow Orchestration

n

Develop and maintain data workflow orchestration frameworks using tools such as Apache Airflow or AWS Step Functions.

n

Automate complex workflows including data ingestion, transformation, validation, and loading processes.

n

Build reusable and configurable workflows to support multiple data processing use cases.

n

Data Migration & Integration

n

Lead data migrations from legacy data warehouse technologies to contemporary AWS data platforms.

n

Perform data migration from RDBMS systems (e.g., MySQL, SQL Server, Oracle) to AWS S3 or AWS Redshift.

n

Design scalable migration frameworks for large datasets with minimal downtime.

n

Integrate data sources from enterprise applications and external systems.

n

Data Security & Governance

n

Implement secure data pipelines using AWS security best practices.

n

Manage access control and data governance using AWS IAM and Lake Formation.

n

Ensure data encryption, access management, and compliance across all data platforms.

n





Performance Optimization & Monitoring

n

Monitor data pipelines and troubleshoot performance issues.

n

Optimize ETL workflows for scalability, reliability, and cost efficiency.

n

Implement logging, monitoring, and alerting mechanisms for data pipelines.

n

Required Skills & Qualifications (Must Have)

n

5+ years of experience in Data Engineering or Data Platform development

n

Strong hands-on experience with:

n

AWS

n

AWS Glue

n

AWS S3

n

AWS Lambda

n

Experience with Data Workflow Orchestration tools such as Apache Airflow or AWS Step Functions

n

Experience performing data migrations from other data warehouse technologies

n

Experience performing data migrations from RDBMS systems to AWS S3 or AWS Redshift

n

Strong expertise in Python and SQL for building scalable data pipelines

n

Solid understanding of ETL/ELT concepts, data partitioning, and distributed data processing

n

Experience working with version control systems such as GitLab or Bitbucket

n

Strong debugging, analytical thinking, and problem-solving skills

n

Basic understanding of Object-Oriented Programming concepts

n

Industry Knowledge & Experience

n

Experience building cloud-native data engineering solutions on AWS

n

Experience with data warehouse architectures and large-scale analytics platforms

n

Hands-on experience with data extraction, transformation, and migration frameworks

n

Experience working in high-volume data environments such as FinTech, analytics platforms, or enterprise data systems

n

Good to Have Skills

n

IBM Cognos

n

AWS Athena

n

AWS Lake Formation

n

AWS Redshift

n

AWS Glue Data Catalog

n

AWS SageMaker

n

AWS IAM

n

Soft Skills

n

Strong communication skills to present technical solutions and recommendations to stakeholders

n

Ability to work cross-functionally in a fast-paced and evolving environment

n

Detail-oriented with a proactive approach to identifying and solving data platform challenges

n

Ability to collaborate effectively with data scientists, analysts, and platform engineering teams

Apply on Kit Job: kitjob.in/job/42qevu

📌 Data Engineer (Nellore)
🏢 Trantor
📍 Nellore