Job DescriptionBig Data Platform Engineering Hands-on experience with Big Data ecosystems, including Hadoop distributions, Apache Spark, and distributed ETL pipelines. Supporting, maintaining, and extending large-scale Big Data platform infrastructure under production workloads. Designing, building, testing, and troubleshooting Big Data environments to sustain high reliability and uptime. Conducting capacity planning and scaling infrastructure as data ingestion volumes and compute workloads expand. Documenting platform configurations, infrastructure standards, operational runbooks, and architectures.
Platform Automation & AI Tooling
• Familiarity with applying AI technologies and automation tools within platform operations and data environments.
• Automating recurring operational tasks, monitoring routines, and administrative runbooks using AI-enabled tooling.
• Robust operational scripting and automation expertise using Python, Bash, or similar languages.
• Evaluating and benchmarking open-source technologies to improve efficiency, performance, and operational cost.

Systems,



Reliability & Observability
• Solid working knowledge of Linux systems administration and OS-level performance tuning.
• Hands-on experience configuring monitoring, log aggregation, and alerting platforms for data infrastructure.
• Monitoring platform health metrics, job execution performance, and hardware/cluster resource utilization.
• Resolving, debugging, and escalating distributed system errors, job failures, and resource contention issues.

Delivery & Engineering Standards

• Working within Agile delivery frameworks to iteratively deliver platform capabilities.
• Researching and benchmarking emerging tooling against existing platform capabilities.
• Prior experience working within banking, financial services, or regulated enterprise environments is preferred.
• Self-motivated approach to identifying gaps and driving platform enhancements with minimal supervision.

Requirements (Must Have

📌 Data Engineer (Bengaluru)
🏢 SmartX Technologies
📍 Bengaluru