Senior Big Data Engineer (AWS | PySpark | EMR)
? Location: Bangalore (Hybrid/Onsite)
? Experience: 10 to 16Years
? Relevant Experience: Minimum 8 Years in Data Engineering / Big Data
Job Description
We are looking for a highly skilled Senior Big Data Engineer with solid hands-on experience in AWS, PySpark/Scala, Apache Spark, EMR, and enterprise-scale data processing.
Candidates should have proven experience building and optimizing large-scale distributed data platforms and must be comfortable working with complex ETL ecosystems handling massive data volumes.
Mandatory Skills
AWS (Must Have)
• Strong hands-on experience with AWS ecosystem
• Expert knowledge of:
• AWS EMR
• S3
• Glue
• Redshift
• Athena
• Lambda
• Step Functions
• CloudWatch
• Experience designing and managing large-scale AWS data platforms

Big Data Technologies (Must Have)
• Apache Spark
• PySpark and/or Scala
• Hadoop Ecosystem
• Hive
• HDFS
• Spark SQL
• Distributed Data Processing

Data Engineering (Must Have)
• Design and development of scalable ETL/ELT pipelines
• Batch and large-scale data processing




• Data Lake and Data Warehouse solutions
• Performance tuning of Spark applications
• Data quality and validation frameworks

Orchestration (Must Have)
• Apache Airflow
• AWS Step Functions
• Workflow orchestration
• DAG design and optimization
• Job scheduling and dependency management
• State management for enterprise data pipelines

Required Experience
• 8+ years of Data Engineering / Big Data experience
• Strong experience handling TB/PB-scale datasets
• Experience working with large Spark clusters
• Expertise in Spark optimization techniques:
• Partitioning
• Caching
• Broadcast joins
• Shuffle optimization
• Memory tuning
• Cluster tuning
• Experience solving distributed computing challenges

Preferred Skills
• Kafka
• Spark Streaming
• Databricks
• Snowflake
• CI/CD Pipelines
• Terraform
• Python
• SQL
• Cloud Migration Projects

Responsibilities
• Build and maintain scalable Big Data pla

📌 Senior Big Data Engineer (Bengaluru)
🏢 ITC Infotech
📍 Bengaluru