13 Nov
|
ElementSkill
|
Pune
About the Role : We are seeking a highly skilled Senior Data Engineer with strong hands-on experience in AWS Cloud, cloud infrastructure, data ingestion, and Python. The ideal candidate will have a proven track record of designing and implementing scalable data platforms, organizing GCP environments to meet data sovereignty requirements, and building secure and reliable data ingestion and transfer solutions.Key Responsibilities : 1. AWS Cloud Engineering : - Work with AWS services such as S3, Glue, Lambda, Redshift, and CloudWatch to build and operate data solutions.- Apply appropriate security, access-control, and cloud data management practices.- Optimize cloud workloads for performance, reliability, and cost.2. Data Ingestion & Pipeline Engineering : - Design and implement robust data ingestion pipelines for structured and unstructured data sources.- Build production-grade ingestion frameworks to reliably process and move data into GCP or similar storage.- Develop scalable and fault-tolerant data pipelines with appropriate validation, monitoring, and error handling.- Optimize data ingestion workflows for performance, reliability, and cost.- Ensure data pipelines maintain data quality, integrity, and traceability throughout the ingestion process.3. Python Development & Automation : - Develop production-grade data engineering applications and automation using Python.- Build reusable libraries, services,
and utilities to support data ingestion and processing workflows.- Automate infrastructure and data engineering processes to improve operational efficiency.- Implement appropriate logging, monitoring, error handling, and testing for Python-based services and pipelines.4. Collaboration & Ownership : - Work closely with Data Engineers, Software Engineers, Cloud/Infrastructure Engineers, Security teams, and business stakeholders.- Take ownership of technical initiatives from architecture and design through production deployment and ongoing support.Requirements : - 5+ years of experience in Data Engineering, Cloud Engineering, or a related field.- Strong hands-on experience with AWS.- Proven experience designing and implementing AWS architecture and infrastructure for data workloads.- Good understanding of AWS services.- Experience with cloud networking, security, authentication, authorization, and access control.- Experience with infrastructure automation and Infrastructure as Code (IaC), preferably Terraform.- Strong understanding of data quality, monitoring, observability, and production operational best practices.- Experience organizing GCP environments to meet data sovereignty and data residency requirements.- Solid experience building data ingestion pipelines and data processing workflows.- Hands-on experience designing and implementing secure data transfer and proxy solutions.- Strong proficiency in Python and experience building production-grade applications and data pipelines.- Excellent communication skills and ability to work independently with minimal supervision. (ref:hirist.tech)
📌 Senior Data Engineer - AWS (Pune)
🏢 ElementSkill
📍 Pune