Job DescriptionSenior Big Data Engineer (AWS | PySpark | EMR)
NLocation: Bangalore (Hybrid/Onsite)
NExperience: 10 to 16Years
NRelevant Experience: Minimum 8 Years in Data Engineering / Big Data
NJob Description
nWe are looking for a highly skilled Senior Big Data Engineer with strong hands-on experience in AWS, PySpark/Scala, Apache Spark, EMR, and enterprise-scale data processing.
NCandidates should have proven experience building and optimizing large-scale distributed data platforms and must be comfortable working with complex ETL ecosystems handling massive data volumes.
NMandatory Skills
nAWS (Must Have)
N
- n
- Solid hands-on experience with AWS ecosystem N
- Expert knowledge of: N
- AWS EMRn
- S3 N
- Gluen
- Redshift N
- Athenan
- Lambda N
- Step Functionsn
- CloudWatch N
- Experience designing and managing large-scale AWS data platforms N

nBig Data Technologies (Must Have)
N
- n
- Apache Sparkn
- PySpark and/or Scala N
- Hadoop Ecosystemn
- Hive N
- HDFSn
- Spark SQL N
- Distributed Data Processing N

nData Engineering (Must Have)




N
- n
- Design and development of scalable ETL/ELT pipelines N
- Batch and large-scale data processing N
- Data Lake and Data Warehouse solutions N
- Performance tuning of Spark applications N
- Data quality and validation frameworks N

nOrchestration (Must Have)
N
- n
- Apache Airflow N
- AWS Step Functionsn
- Workflow orchestration N
- DAG design and optimization N
- Job scheduling and dependency management N
- State management for enterprise data pipelines N

nRequired Experience
N
- n
- 8+ years of Data Engineering / Big Data experience N
- Strong experience handling TB/PB-scale datasets N
- Experience working withlarge Spark clusters N
- Expertise in Spark optimization techniques: N
- Partitioningn
- Caching N
- Broadcast joinsn
- Shuffle optimization N
- Memory tuningn
- Cluster tuning N
- Experience solving distributed computing challenges N

nPrefer

📌 Senior Big Data Engineer (Karnataka)
🏢 ITC Infotech
📍 Karnataka