Job Information

Date Opened

07/23/2025

Industry

Information Technology

Job Type

Full time

Work Experience

5+ years

City

Hyderabad

State/Province

Telangana

Country

India

Zip/Postal Code

500039

Job Description

Core Responsibilities

- Design and optimize batch/streaming data pipelines using Scala, Spark, and Kafka

- Implement real-time tokenization/cleansing microservices in Java

- Manage production workflows via Apache Airflow (batch scheduling)

- Conduct root-cause analysis of data incidents using Spark/Dynatrace logs

- Monitor EMR clusters and optimize performance via YARN/Dynatrace metrics

- Ensure data security through HashiCorp Vault (Transform Secrets Engine)

- Validate data integrity and configure alerting systems

Requirements

- Programming :Scala (Spark batch/streaming), Java (real-time microservices)

- Big Data Systems: Apache Spark, EMR, HDFS, YARN resource management





- Cloud & Storage :Amazon S3, EKS

- Security: HashiCorp Vault, tokenization vs. encryption (FPE)

- Orchestration :Apache Airflow (batch scheduling)

- Operational Excellence Spark log analysis, Dynatrace monitoring, incident handling, data validation

Mandatory Competencies

- Expertise in distributed data processing (Spark on EMR/Hadoop)

- Proficiency in shell scripting and YARN job management

- Ability to implement format-preserving encryption (tokenization solutions)

- Experience with production troubleshooting (executor logs, metrics, RCA)

Benefits

Advantages

Insurance - Family
Term Insurance
PF

Paid Time Off - 20 days
Holidays - 10 days
Flexi timing
Competitive Salary
Diverse & Inclusive workspace

📌 Big Data Engineer (India)
🏢 Kmccorp India
📍 India