Job DescriptionSenior Big Data Engineer (AWS | PySpark | EMR)

NLocation: Bangalore (Hybrid/Onsite)

NExperience: 10 to 16Years

NRelevant Experience: Minimum 8 Years in Data Engineering / Big Data

NJob Description

nWe are looking for a highly skilled Senior Big Data Engineer with robust hands-on experience in AWS, PySpark/Scala, Apache Spark, EMR, and enterprise-scale data processing.

NCandidates should have proven experience building and optimizing large-scale distributed data platforms and must be comfortable working with complex ETL ecosystems handling massive data volumes.

NMandatory Skills

nAWS (Must Have)

N

n
- Strong hands-on experience with AWS ecosystem
N
- Expert knowledge of:
N
- AWS EMR
n
- S3
N
- Glue
n
- Redshift
N
- Athena
n
- Lambda
N
- Step Functions
n
- CloudWatch
N
- Experience designing and managing large-scale AWS data platforms
N

nBig Data Technologies (Must Have)

N

n
- Apache Spark
n
- PySpark and/or Scala
N
- Hadoop Ecosystem
n
- Hive
N
- HDFS
n
- Spark SQL
N
- Distributed Data Processing
N

nData Engineering (Must Have)

N

n
- Design and development of scalable ETL/ELT pipelines
N
- Batch and large-scale data processing
N
- Data Lake and Data Warehouse solutions
N
- Performance tuning of Spark applications
N
- Data quality and validation frameworks
N

nOrchestration (Must Have)

N

n
- Apache Airflow
N




- AWS Step Functions
n
- Workflow orchestration
N
- DAG design and optimization
N
- Job scheduling and dependency management
N
- State management for enterprise data pipelines
N

nRequired Experience

N

n
- 8+ years of Data Engineering / Big Data experience
N
- Strong experience handling TB/PB-scale datasets
N
- Experience working withlarge Spark clusters
N
- Expertise in Spark optimization techniques:
N
- Partitioning
n
- Caching
N
- Broadcast joins
n
- Shuffle optimization
N
- Memory tuning
n
- Cluster tuning
N
- Experience solving distributed computing challenges
N

nPreferred Skills

N

n
- Kafka
n
- Spark Streaming
N
- Databricks
n
- Snowflake
N
- CI/CD Pipelines
n
- Terraform
N
- Python
n
- SQL
N
- Cloud Migration Projects
N

nResponsibilities

N

n
- Build and maintain scalable Big Data platforms on AWS
N
- Develop and optimize PySpark/Scala applications
N
- Design enterprise-gradeETL/ELT frameworks
N
- Create and maintain Airflow DAGs and orchestration workflows
N
- Optimize Spark jobs andEMR cluster performance
N
- Implement monitoring, alerting, and reliability solutions
N
- Work closely with business and analytics teams to deliver data solutions
N
- Troubleshoot productionissues and improve platform stability
N

📌 Hiring: Senior Big Data Engineer (Bengaluru)
🏢 ITC Infotech
📍 Bengaluru