Role-Senior Data Engineer

Experience - 7 to 15 years

Location- Pune, Hyderabad, Bangalore

Required Skills & Experience

Must Have

- 7-12 years of data engineering experience, with strong recent hands-on responsibility for distributed batch or streaming pipelines.
- Strong hands-on Apache Spark experience using PySpark, Scala, or Spark SQL, including tuning and production troubleshooting.
- Experience building lakehouse pipelines on Google Cloud Storage using Apache Iceberg or a comparable open table format.
- Strong SQL skills with practical experience in profiling, transformations, joins, aggregations, incremental processing, and reconciliation.
- Experience with Apache Kafka and Apache Flink or Spark Structured Streaming.
- Hands-on Apache Airflow experience covering scheduling, dependencies, retries, backfills, and monitoring.
- Experience with database, file, API, event, and CDC ingestion using Kafka Connect, Debezium, Datastream, or comparable tools.
- Understanding of data modeling, schema evolution, data contracts, partitioning,



file formats, data quality, lineage, and governance.
- Working knowledge of Git, automated testing, CI/CD, containers, GKE-based execution, Cloud Storage, Workload Identity Federation, and Secret Manager.
- Strong problem solving, documentation, collaboration, and stakeholder communication skills.

Positive to Have
- Experience with Polaris, Nessie, REST catalogs, Trino, OpenMetadata, schema registries, Dataproc, or data-quality frameworks.
- Experience with GCP data services and regulated enterprise environments.
- Exposure to banking data, high-volume transaction processing, reconciliation, auditability, or data privacy controls.
- Experience with reusable data-product patterns, data contracts, semantic models, or domain-oriented data platforms.
- Exposure to performance engineering, recovery testing, and multi-tenant GCP data platforms.

📌 Senior Data Engineer (Bengaluru)
🏢 Tata Consultancy Services
📍 Bengaluru