19 Nov
|
IDESLABS
|
Bengaluru
19 Nov
IDESLABS
Bengaluru
Job Description Big Data Engineer (AWS, Snowflake & Python) Role: Big Data Engineer Experience: 6+ Years
Location: Bangalore
Employment Type: Full-Time Job Summary We are seeking a highly skilled Big Data Engineer with strong expertise in AWS, Snowflake, Python, and Big Data technologies. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines, data lakes, and warehousing solutions. The candidate should have hands-on experience with cloud-based data platforms and large-scale data processing frameworks. Key Responsibilities
Design, develop, and maintain scalable ETL/ELT pipelines.
Build and optimize data ingestion frameworks from multiple data sources.
Develop and support data lake and data warehouse solutions using Snowflake.
Implement large-scale data processing solutions using PySpark/Spark.
Work closely with Data Architects, Analysts, and Business teams to deliver data solutions.
Develop automation and orchestration workflows for data processing.
Ensure data quality, governance, security, and performance optimization.
Monitor and troubleshoot production data workloads.
Mandatory Skills
Robust experience in Python
Hands-on expertise in AWS Cloud
Extensive experience with Snowflake
Solid knowledge of PySpark/Spark
Experience in ETL/ELT Development
Robust SQL programming skills
Experience with Data Warehousing and Data Lakes
Experience with Apache Hadoop/Hive
Knowledge of Kafka or streaming technologies
Preferred Skills
Experience with Databricks
Knowledge of Airflow, Glue, EMR, Lambda
Experience with CI/CD pipelines
Exposure to Agile/Scrum methodologies
AWS Services Preferred
AWS S3
AWS Glue
AWS EMR
AWS Lambda
AWS Redshift
AWS IAM
CloudWatch
Mandatory Screening Keywords AWS, Snowflake, Python, PySpark, Spark, ETL, SQL, Data Warehouse, Data Lake, Hadoop, Hive, Kafka, Databricks, AWS Glue, EMR Ideal Profile Candidates currently working as:
Big Data Engineer
Data Engineer
Senior Data Engineer
Cloud Data Engineer
Snowflake Data Engineer
AWS Data Enginee
📌 Big Data Bengaluru
🏢 IDESLABS
📍 Bengaluru