Job Summary

A Senior Data Platform Engineer with 12-14 years of experience to own and build the core data platform for a multi-region CPaaS system supporting multiple digital channels.

The platform processes 40,000+ transactions per second, with multiple (10) state updates per transaction, and powers real-time metrics, billing, reporting, campaign analytics, and audits.

This role is hands-on and ownership-driven, focused on data correctness, scalability, and reliability in large-scale streaming systems.

Responsibilities Mandatory / Primary Responsibilities

- Own the end-to-end data architecture for high-throughput, event-driven CPaaS systems.
- Design state-transition and event-sourced data models handling millions of updates per second.
- Build and operate streaming data pipelines using Apache Pulsar for ingestion and Apache Flink for stateful processing and real-time aggregation.
- Design and maintain a data lake using Apache Hudi to support: Upserts and deletes
- Incremental queries
- Time-travel and auditability
- Build real-time analytics datasets in ClickHouse for dashboards, tenant metrics, and campaign analysis.
- Define and enforce processing semantics, including idempotency, deduplication, ordering, and replay safety.
- Own data correctness, completeness, and freshness SLAs across streaming and analytics systems.
- Design multi-region data ingestion and aggregation, including replication, failover, and reprocessing.
- Lead hands-on development, code reviews, performance tuning, and production troubleshooting.

Secondary Responsibilities
- Mentor senior engineers and review architectural designs.
- Define data standards, schema evolution practices, and platform guidelines.




- Participate in release planning and technical prioritization.
- Evaluate new data technologies and patterns through POCs.
- Collaborate with product and operations teams on data-driven features.

Skills Mandatory / Primary Skills
- Apache Pulsar (topics, partitions, subscriptions, geo-replication)
- Apache Flink (stateful stream processing, windows, checkpoints)
- Apache Hudi (Merge-On-Read, Copy-On-Write, incremental processing)
- ClickHouse (real-time analytics, high-cardinality datasets)
- Event-driven and streaming data architectures
- Event sourcing and state-transition modeling
- Idempotent replay-secure data pipelines
- Designing systems for high TPS and frequent state updates
- Multi-region data design and disaster recovery

Secondary Skills
- Redis for low-latency caching, counters, and ephemeral state
- PostgreSQL for transactional, relational, and reference data
- MySQL for operational data stores and legacy integrations
- Low-Level Design (LLD) for scalable, fault-tolerant components
- High-Level Design (HLD) for distributed, multi-region systems
- NoSQL stores for hot state (e.g., Mongo, DynamoDB, Cassandra)
- Relational databases for reference/config data
- Data lake query engines (Athena, Spark)
- Observability for data pipelines (lag, freshness, reconciliation)
- Containerized runtimes and cloud-native deployments
- Prior experience with CPaaS, telecom, or large-scale messaging platforms

Experience Background
- 12-14 years of experience in data engineering, platform engineering, or distributed systems
- Proven experience building large-scale, streaming-heavy data platforms
- Strong hands-on background in real-time analytics and data consistency
- Comfortable operating at Principal Staff Engineer level with full technical ownership

📌 Sr Manager - Customer Interaction Suite (Chennai)
🏢 Tata Communications
📍 Chennai