? Work Mode: Remote ? Employment Type: Part-Time | Freelance About the Chance We are looking for a experienced AWS EMR / Apache Spark Infrastructure Engineer to provide technical guidance and hands-on troubleshooting support for an enterprise-scale DRAM memory optimization project. The ideal candidate will have strong expertise in Amazon EMR cluster architecture, Apache Spark memory management, distributed workload optimization, and AWS infrastructure right-sizing. You will help identify memory bottlenecks, optimize cluster configurations, evaluate AWS Graviton-based instances, and improve infrastructure efficiency while maintaining performance and reliability.

Key Responsibilities
• Analyze AWS EMR cluster architecture, resource allocation, and memory consumption.
• Optimize Apache Spark executor sizing, driver memory, memory overhead, and dynamic allocation.
• Troubleshoot OutOfMemoryError, executor failures, garbage collection issues, and memory spills.
• Evaluate AWS EC2 instance families, Graviton/ARM64 compatibility, and cluster right-sizing opportunities.
• Identify opportunities to reduce DRAM consumption and infrastructure costs across distributed workloads.
• Monitor Spark UI metrics, Amazon CloudWatch, EMR logs, CPU utilization, and executor performance.
• Design and execute controlled performance optimization experiments in non-production environments.
• Compare baseline and optimized configurations using measurable memory, throughput, latency, and cost metrics.




• Validate performance improvements without compromising workload reliability or availability.
• Provide technical guidance on configuration changes, deployment strategies, rollback procedures, and production-safe optimization.
• Prepare technical documentation, optimization recommendations, runbooks, and SOPs. Required Technical Skills
• 5+ years of strong hands-on experience with Amazon EMR infrastructure and cluster optimization.
• Advanced understanding of Apache Spark architecture and memory management.
• Expertise in executor and driver memory tuning, memory overhead, garbage collection, and OOM troubleshooting.
• Experience with Spark Dynamic Allocation and executor scaling.
• Experience evaluating AWS Graviton / ARM64-based EC2 instances for distributed workloads.
• Strong knowledge of AWS EC2, S3, IAM, CloudWatch, and infrastructure monitoring.
• Proficiency in Spark UI, EMR logs, performance metrics, and root-cause analysis.
• Understanding of JVM memory management and large-scale distributed systems.
• Experience with PySpark, Spark SQL, Scala, or Java.
• Linux administration and shell scripting skills. Preferred Qualifications
• Experience with enterprise-scale big data platforms and production Spark workloads.
• Infrastructure automation and configuration management experience.
• Proven ability to conduct performance benchmarking and infrastructure cost optimization.
• Strong analytical, troubleshooting, communication, and technical documentation skills.

📌 Freelance Senior AWS EMR (India)
🏢 IndiTech Consulting
📍 India