Senior Big Data Engineer (AWS | PySpark | EMR)? Location: Bangalore (Hybrid/Onsite)? Experience: 10 to 16Years? Relevant Experience: Minimum 8 Years in Data Engineering / Big DataJob DescriptionWe are looking for a highly skilled Senior Big Data Engineer with solid hands-on experience in AWS, PySpark/Scala, Apache Spark, EMR, and enterprise-scale data processing.Candidates should have proven experience building and optimizing large-scale distributed data platforms and must be comfortable working with complex ETL ecosystems handling massive data volumes.Mandatory SkillsAWS (Must Have)Strong hands-on experience with AWS ecosystemExpert knowledge of:AWS EMRS3GlueRedshiftAthenaLambdaStep FunctionsCloudWatchExperience designing and managing large-scale AWS data platformsBig Data Technologies (Must Have)Apache SparkPySpark and/or ScalaHadoop EcosystemHiveHDFSSpark SQLDistributed Data ProcessingData Engineering (Must Have)Design and development of scalable ETL/ELT pipelinesBatch and large-scale data processingData Lake and Data Warehouse solutionsPerformance tuning of Spark applicationsData quality and validation frameworksOrchestration (Must Have)Apache AirflowAWS Step FunctionsWorkflow orchestrationDAG design and optimizationJob scheduling and dependency managementState management for enterprise data pipelinesRequired Experience8+ years of Data Engineering / Big Data experienceStrong experience handling TB/PB-scale datasetsExperience working with large Spark clustersExpertise in Spark optimization techniques:PartitioningCachingBroadcast joinsShuffle optimizationMemory tuningCluster tuningExperience solving distributed computing challengesPreferred SkillsKafkaSpark StreamingDatabricksSnowflakeCI/CD PipelinesTerraformPythonSQLCloud Migration ProjectsResponsibilitiesBuild and maintain scalable Big Data platforms on AWSDevelop and optimize PySpark/Scala applicationsDesign enterprise-grade ETL/ELT frameworksCreate and maintain Airflow DAGs and orchestration workflowsOptimize Spark jobs and EMR cluster performanceImplement monitoring, alerting, and reliability solutionsWork closely with business and analytics teams to deliver data solutionsTroubleshoot production issues and improve platform stability

📌 Senior Big Data Engineer (Bengaluru)
🏢 ITC Infotech
📍 Bengaluru