About the Role:
Grade Level (for internal use):
12Key Responsibilities
Data Pipeline Transition and Platform Delivery
Partner with the core Databricks team to plan and execute the transition of existing data pipelines to the target data platform.
Drive the implementation of repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring across onboarded datasets.
Ensure pipelines are designed and managed in a way that supports long-term platform consistency, reliability, observability, and ease of support.
Guide the design and operation of cloud-native data pipelines using AWS services such as:
Amazon S3 for durable data lake storage
AWS Glue for integration, cataloging, and processing
AWS Lambda for event-driven processing
Amazon Kinesis for streaming use cases
AWS Lake Formation for governed data lake controls
Promote the use of AWS IAM, encryption, workplace-level controls, and platform guardrails to enforce secure access to platform resources and data products.
Support practical application of lakehouse technologies and concepts such as Delta Lake, Apache Iceberg, Databricks Unity Catalog, metadata-driven pipelines, and governed data access patterns.
Data Onboarding and Asset-Agnostic Enablement
Define and operationalize onboarding patterns that support a broad range of data assets, domains, and source systems without requiring bespoke platform redesign for each use case.
Work with platform, data engineering, architecture, governance, and business-aligned teams to simplify and standardize how data is ingested, transformed, governed, and published to the enterprise platform.
Create or contribute to reusable technical assets such as design patterns, reference implementations, onboarding templates, pipeline frameworks, technical documentation, and operational runbooks.
Support asset-agnostic onboarding by ensuring data pipelines are configurable, metadata-driven, scalable, and aligned with enterpr
📌 Associate Director, Data Platforms — Technical Lead Hyderabad (India)
🏢 Osttra
📍 India