Role- Lead Data Platform Engineer
Experience- 8 to 15 years
Location- Pune, Hyderabad, Bangalore
Required Skill set- Spark, Kafka, Flink, Airflow, Iceberg, Polaris/Nessie, Kubernetes

Required Skills & Experience
Must Have
• 8-15 years of data engineering or data platform experience, with substantial recent hands-on responsibility for distributed processing platforms.
• Deep hands-on experience with Apache Spark, including DataFrame or SQL workloads, performance tuning, packaging, configuration, and production troubleshooting.
• Robust Apache Kafka experience covering topics, partitions, consumer groups, offsets, delivery semantics, schemas, security, and operations.
• Practical Apache Flink experience covering state, checkpoints, savepoints, event time, recovery, and backpressure.
• Strong understanding of Apache Iceberg, Google Cloud Storage, schema and partition evolution, snapshots, compaction, and table maintenance.
• Experience with Apache Polaris, Project Nessie, or REST-based Iceberg catalogs.




• Hands-on experience with Apache Airflow for distributed data-processing orchestration.
• Working knowledge of GKE-based execution, containers, Helm, service accounts, Workload Identity Federation, secrets, networking, storage, and observability.
• Proficiency in Python, Java, or Scala, SQL, Git, automated testing, CI/CD, and reusable software engineering practices.
• Strong technical leadership, troubleshooting, documentation, mentoring, and stakeholder communication skills.

Good to Have
• Experience with Trino, OpenMetadata, JupyterHub, Dataproc, data-quality frameworks, lineage, or schema registries.
• Exposure to Debezium, Kafka Connect, Datastream, database logs, or comparable CDC patterns.
• Experience with Spark Operator, Flink Kubernetes Operator, Strimzi, KubernetesExecutor, or operator-based GKE integration.
• Understanding of data security, encryption, masking, governance, data contracts, and regulated enterprise controls.

Experience with multi-tenant dat

📌 Lead Data Platform Engineer (Hyderabad)
🏢 Tata Consultancy Services
📍 Hyderabad