What You''ll Do
Own and evolve our Databricks Lakehouse platform Unity Catalog governance, Delta Lake table management, and Lakeflow Declarative Pipelines.
Design and build reliable, scalable data pipelines using Apache Spark and dbt for batch and near-real-time processing.
Stream and ingest data at scale via Kafka; orchestrate workflows with Airflow.
Partner with the integrations team to onboard current data sources and support AI/ML and analytics use cases.
Own data quality and observability end-to-end using Datadog; proactively identify and resolve discrepancies across data layers.
Manage infrastructure as code using Terraform; ensure platform reliability and cost efficiency on AWS.
Build and maintain MongoDB and PostgreSQL data stores; define schema standards and access patterns.
Deliver data products surfaced in Tableau; collaborate with analytics and product teams on reporting requirements.
Mentor engineers, lead code reviews, and drive engineering best practices across the team.
Participate in hiring and onboarding of recent team members. Required




Deep hands-on expertise with Databricks Unity Catalog, Delta Lake, and Lakeflow Declarative Pipelines.
Robust Apache Spark experience tuning, optimization, and production operations.
Proficiency in Scala (required) and Python for data pipeline development.
Experience with Kafka for real-time data ingestion and streaming architectures.
Solid dbt skills for SQL-based transformation and data modeling.
Familiarity with Airflow for workflow orchestration.
Infrastructure experience with Terraform and AWS (S3, IAM, networking fundamentals).
Experience with MongoDB and PostgreSQL for operational and analytical workloads.
Strong communication skills able to translate complex technical concepts for non-technical audiences.

Keywords: Data Platform: Databricks (Unity Catalog, Delta Lake, Lakeflow Declarative Pipelines), Apache Spark

Databases: MongoDB, PostgreSQL

Cloud: AWS (S3, EMR, Lambda, Glue, IAM, VPC)

Streaming & Messaging: Apache Kafka

Orchestration: Apache Airflow

📌 Sr Data Engineer Hitec City (India)
🏢 Zeta Global
📍 India