Job Overview We are seeking an experienced Cloud Data Engineer to the architecture, development, Data migration, ETL pipeline in cloud and optimization of a next-generation data platform. This is a development role for an individual with over 4 years of deep data engineering and data migration, data pipeline in AWS expertise, capable of driving technical direction, and delivering high-impact solutions in a fast-paced Key Responsibilities Platform Development

Knowledge of best practices for our end-to-end data pipelines in Cloud, ensuring scalability, reliability, and security across the platform. Aggressively folowing the project timeline to ensure rapid delivery of data migratation in AWS RDS. Working with Data Analysts, and business stakeholders to translate complex requirements into robust, production-ready ETL data solutions.

Data Pipeline

Development

Ingestion & Transformation: Design, build, and optimize high-volume data ingestion and transformation jobs using tools like dbt Core, iceberg, AWS Glue, ensuring data quality and integrity. Building ELT/ETL pipelines and using workflow orchestration tools like Apache Airflow or dbt Optimize large-scale data processing algorithms and cloud data warehouse performance.

Workflow: Develop and maintain sophisticated data pipelines using AWS glue/lambda, focusing on modularity and reusability. Streaming & Real-time Integration: Implement and manage real-time data flows utilizing native AWS streaming services (e.g., Kinesis)



for immediate data availability.

Data Security and Privacy: Data Anonymization, Compliance with Regulations

Good knowledge of CI/CD for ETL pipeline automation and Data migration Drive the Data Ecosystem Management & Monitoring

Open Table Formats & Management: Implement and maintain the Iceberg open table format, utilizing tools for efficient schema evolution and data management.

Compute Engine Optimization: Optimize query performance and cost efficiency across our primary compute engines: Amazon Redshift, and AWS Athena. rapidly identify bottlenecks, and troubleshoot production issues Candidate Profile 4+ Years of hands-on, progressive experience in Cloud Data Engineering, cloud ecosystems AWS, Data Architecture Deep conceptual understanding of core data engineering principles, ETL/ELT patterns, and metadata management  Ability to translate complex business needs into actionable technical requirements.

 Solid troubleshooting and problem-solving skills in distributed systems.

 Agile/Scrum teamwork capabilities and version control practices

Insurance industry experience preferred but not mandatory

Tools

Cloud Environment: AWS (S3, IAM, VPC, Glue, Lambda etc.)

Experience with dbt Core, Iceberg, AWS Glue Catalog, AWS lambda, Redshift, Athena, Splunk, AmazonQ, AWS streaming services, Git Strong SQL, Pyspark and Python Skills: pipeline,etl,aws,dbt,data,management,cloud

📌 AWS Data engineer (Pune)
🏢 CG-VAK Software u0026 Exports
📍 Pune