Senior Big Data Engineer (AWS | PySpark | EMR)
Location: Bangalore (Hybrid/Onsite)
Experience:

10 to 16Years
Relevant Experience:

Minimum 8 Years in Data Engineering / Big Data
Job Description
We are looking for a highly skilled

Senior Big Data Engineer

with robust hands-on experience in

AWS, PySpark/Scala, Apache Spark, EMR, and enterprise-scale data processing .
Candidates should have proven experience building and optimizing large-scale distributed data platforms and must be comfortable working with complex ETL ecosystems handling massive data volumes.
Mandatory Skills
AWS (Must Have)
Strong hands-on experience with AWS ecosystem
Expert knowledge of:
AWS EMR
S3
Glue
Redshift
Athena
Lambda
Step Functions
CloudWatch
Experience designing and managing large-scale AWS data platforms
Big Data Technologies (Must Have)
Apache Spark
PySpark and/or Scala
Hadoop Ecosystem
Hive
HDFS
Spark SQL
Distributed Data Processing
Data Engineering (Must Have)
Design and development of scalable ETL/ELT pipelines
Batch and large-scale data processing
Data Lake and Data Warehouse solutions




Performance tuning of Spark applications
Data quality and validation frameworks
Orchestration (Must Have)
Apache Airflow
AWS Step Functions
Workflow orchestration
DAG design and optimization
Job scheduling and dependency management
State management for enterprise data pipelines
Required Experience
8+ years of Data Engineering / Big Data experience
Strong experience handling

TB/PB-scale datasets
Experience working with large Spark clusters
Expertise in Spark optimization techniques:
Partitioning
Caching
Broadcast joins
Shuffle optimization
Memory tuning
Cluster tuning
Experience solving distributed computing challenges
Preferred Skills
Kafka
Spark Streaming
Databricks
Snowflake
CI/CD Pipelines
Terraform
Python
SQL
Cloud Migration Projects
Responsibilities
Build and maintain scalable Big Data platforms on AWS
Develop and optimize PySpark/Scala applications
Design enterprise-grade ETL/ELT frameworks

📌 Senior Big Data Engineer (Vasanth Nagar)
🏢 ITC Infotech
📍 Vasanth Nagar