06 Sep
|
Facctum
|
Bengaluru
Job Description
We are seeking an experienced AWS Data Engineer to lead and execute a large-scale migration from Matillion ETL to a up-to-date AWS-native data platform. The ideal candidate will possess deep expertise in Apache Airflow, AWS Glue, AWS Lambda and Snowflake, with a strong background in designing scalable data pipelines, cloud-based data integration, workflow orchestration, and data warehouse modernization.
Total Experience : 2 - 5 years
Key Responsibilities
Develop modules/code for migration of existing Matillion jobs to AWS-native solutions.
Participate in Requirements Analysis of existing data processing jobs.
Design, develop, and promote scalable data pipelines using AWS Glue and Apache Airflow.
Build robust data ingestion, transformation, and orchestration frameworks on AWS.
Create reusable and modular ETL/ELT components for enterprise data platforms.
Implement data quality, monitoring, alerting, security,
governance, and compliance standards.
Collaborate with data architects and business teams to understand migration requirements and translate them into technical solutions.
Optimize data processing performance, cost, and scalability across AWS services.
Participate in CI/CD best practices and DevOps processes for data engineering workflows.
Tech Stack
Apache Airflow (DAG development, workflow orchestration, scheduling, monitoring)
AWS Glue (ETL jobs, Glue Studio, Crawlers, Data Catalog, Spark-based transformations)
Snowflake (Data Warehousing, SnowSQL, Snowpipe, Streams & Tasks, Performance Tuning)
AWS Lambda and Step Functions
Programming language s: Python, SQL, Spark
Git experience
(ref:hirist.tech)
📌 Facctum - AWS Data Engineer (Bengaluru)
🏢 Facctum
📍 Bengaluru