Description

- 4+ Years of hands-on, progressive experience in Cloud Data Engineering, Data Architecture
- Deep conceptual understanding of core data engineering principles, ETL/ELT patterns, and metadata management
- Insurance industry experience preferred but not mandatory
- Tools:
- Cloud Workplace: AWS (S3, IAM, VPC, Glue, Lambda etc.)
- Experience with dbt Core, Iceberg, AWS Glue Catalog, AWS lambda, Redshift, Athena, Splunk, AmazonQ, AWS streaming services, Git
- Strong SQL, Pyspark and Python

Responsibilities

- Platform Development
- Knowledge of best practices for our end-to-end data pipelines in Cloud, ensuring scalability, reliability, and security across the platform.
- Aggressively folowing the project timeline to ensure rapid delivery of data migration in AWS RDS.
- Working with Data Analysts, and business stakeholders to translate complex requirements into robust, production-ready ETL data solutions.
- Architect and maintain reliable data pipelines to make raw data usable for analysts and data scientists.
- Optimize large-scale data processing algorithms and cloud data warehouse performance.
- Ensure data security, governance, and access control across cloud environments

- Data Pipeline Development

- Ingestion & Transformation: Design, build, and optimize high-volume data ingestion and transformation jobs using tools like dbt Core, iceberg, AWS Glue, ensuring data quality and integrity.
- Workflow:



Develop and maintain sophisticated data pipelines using AWS glue/lambda, focusing on modularity and reusability.
- Streaming & Real-time Integration: Implement and manage real-time data flows utilizing native AWS streaming services (e.g., Kinesis) for immediate data availability.
- Data Security and Privacy: Data Anonymization, Compliance with Regulations
- Positive knowledge of CI/CD for ETL pipeline automation and Data migration

- Drive the Data Ecosystem Management & Monitoring

- Open Table Formats & Management: Implement and maintain the Iceberg open table format, utilizing tools for efficient schema evolution and data management.
- Compute Engine Optimization: Optimize query performance and cost efficiency across our primary compute engines: Amazon Redshift, and AWS Athena.
- rapidly identify bottlenecks, and troubleshoot production issues

Qualifications

B. Tech or MCA with 4+ years of cloud data engineering experience. Hands-on experience with cloud ecosystems AWS, Redshift, SQL, Python, Pyspark, Data migration into Cloud, pipeline building, CI/CD pipeline automation. Glue, Lambda Amazon Q or Codex AI. Good, comunication Skill, Building ELT/ETL pipelines and using workflow orchestration tools like Apache Airflow or dbt. Ability to translate complex business needs into actionable technical requirements. Strong troubleshooting and problem-solving skills in distributed systems. Agile/Scrum teamwork capabilities and version control practices

📌 Assistant Manager/ Consultant 2 (Gurugram)
🏢 EXL
📍 Gurugram