Job Description

About the Role

We are looking for a Data Platform Engineer (IC2) to build and operate scalable, reliable, and high-performance data platforms. In this role, you will develop streaming and batch data pipelines, manage Databricks environments, and support our event-driven data infrastructure using Kafka and Kafka Connect.

This is an excellent opportunity for engineers with a robust foundation in distributed data processing who want to work on up-to-date data engineering technologies at scale.

Key Responsibilities

- Develop, maintain, and optimize batch and streaming data pipelines using Apache Spark and Apache Flink.
- Build scalable ETL/ELT workflows for processing large datasets.
- Manage and maintain Databricks workspaces, clusters, jobs, notebooks, and workflows.
- Monitor, troubleshoot, and optimize Databricks jobs for performance and cost efficiency.
- Work with Apache Kafka for event-driven data processing and streaming applications.
- Configure, deploy, and troubleshoot Kafka Connect connectors for integrating various data sources and sinks.
- Ensure data reliability through monitoring, alerting,



and operational best practices.
- Collaborate with software engineers, platform engineers, and data consumers to design robust data solutions.
- Participate in production support, incident response, and root cause analysis.
- Contribute to automation, documentation, and continuous improvement of the data platform.

What You'll Learn

- Design and operate enterprise-scale data platforms.
- Build high-throughput streaming applications using Spark and Flink.
- Manage production Databricks environments efficiently.
- Develop expertise in Kafka ecosystem components, including Kafka Connect.
- Work with modern cloud-native data engineering technologies and best practices.

Success Metrics

- Deliver reliable and scalable batch and streaming pipelines.
- Maintain high platform availability and operational excellence.
- Optimize job performance and infrastructure costs.
- Improve data quality, observability, and platform automation.
- Collaborate effectively across engineering teams to deliver business value.

📌 Specialist - Data Platform Engineering (Chennai)
🏢 FreshWorks
📍 Chennai