Experience:
Minimum 8 Years in Data Engineering / Big Data
Job Description
We are looking for a highly skilled Senior Big Data Engineer with strong hands-on experience in AWS, PySpark/Scala, Apache Spark, EMR, and enterprise-scale data processing. Candidates should have proven experience building and optimizing large-scale distributed data platforms and must be comfortable working with complex ETL ecosystems handling massive data volumes. Mandatory Skills AWS (Must Have)
• Solid hands-on experience with AWS ecosystem
• Expert knowledge of:
• AWS EMR
• S3
• Glue
• Redshift
• Athena
• Lambda
• Step Functions
• CloudWatch
• Experience designing and managing large-scale AWS data platforms Big Data Technologies (Must Have)
• Apache Spark
• PySpark and/or Scala
• Hadoop Ecosystem
• Hive
• HDFS
• Spark SQL
• Distributed Data Processing Data Engineering (Must Have)
• Design and development of scalable ETL/ELT pipelines
• Batch and large-scale data processing
• Data Lake and Data Warehouse solutions
• Performance tuning of Spark applications
• Data quality and validation frameworks Orchestration (Must Have)
• Apache Airflow
• AWS Step Functions
• Workflow orchestration
• DAG design and optimization
• Job scheduling and dependency management
• State management for enterprise data pipelines Required Experience
• 8+ years of Data Engineering / Big Data experience
• Strong experience handling TB/PB-scale datasets
• Experience working with large Spark clusters
• Expertise in Spark optimization techniques:
• Partitioning
• Caching
• Broadcast joins
• Shuffle optimization
• Memory tuning
• Cluster tuning
• Experience solving distributed computing challenges Preferred Skills
• Kafka
• Spark Streaming
• Databricks
• Snowflake
• CI/CD Pipelines
• Terraform
• Python
• SQL
• Cloud Migration Projects Responsibilities
• Build and maintain scalable Big Data platforms on AWS
• Develop and optimize PySpark/Scala applications
• Design enterprise-grade ETL/ELT frameworks
• Create and maintain Airflow DAGs and orchestration workflows
• Optimize Spark jobs and EMR cluster performance
• Implement monitoring, alerting, and reliability solutions
• Work closely with business and analytics teams to deliver data solutions
• Troubleshoot production issues and improve platform stability
📌 Senior Big Data Engineer (Bengaluru)
🏢 ITC Infotech
📍 Bengaluru
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.