Job Summary

About the Role: Grade Level (for internal use): 10

Responsibilities Data Pipeline Transition and Platform Delivery

- Partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform.
- Provide hands-on engineering support to stabilize and optimize pipelines post-transition, focusing on performance, scalability, maintainability, and operational reliability.
- Implement and follow repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring.
- Support secure AWS deployment patterns and governed data access using AWS IAM, encryption, and platform guardrails.
- Work with lakehouse technologies such as Delta Lake, Apache Iceberg, and Databricks Unity Catalog, and support metadata-driven pipeline design.
- Assist with establishing and maintaining observability practices (logging, telemetry, alerts, dashboards) for production pipelines.

Data Onboarding and Asset-Agnostic Enablement

- Support asset-agnostic onboarding by applying standard ingestion and transformation frameworks to new datasets.
- Help offshore teams adopt common onboarding standards, delivery practices, and technical templates.
- Contribute to reusable technical assets such as reference implementations, onboarding templates, documentation, and operational runbooks.
- Support both batch and streaming onboarding patterns using AWS services (e.g., AWS Glue, AWS Lambda, Amazon Kinesis) or event-driven integrations as appropriate.

Data Mastering Platform Integration

- Support integration of platform datasets with the enterprise data mastering platform.
- Help establish and/or maintain data quality controls, reconciliation, and metadata alignment for trusted mastered data.
- Support golden record / golden copy generation and maintenance activities for key financial datasets (as applicable to the program).





Technical Leadership and Engineering Excellence

- Provide technical guidance to engineering peers through code reviews, design reviews, and implementation planning.
- Contribute to engineering standards and delivery rigor across CI/CD, infrastructure automation, testing practices, and operational readiness.
- Help ensure documentation quality and operational discipline (runbooks, support procedures, incident learnings).
- Act as a technical bridge between offshore execution teams and global platform stakeholders to resolve delivery blockers and keep alignment.

AI-Assisted Engineering and Context Engineering

- Use AI tools (Claude, GitHub CoPilot, LLMs) to accelerate engineering tasks such as drafting code, refactoring, writing documentation, generating unit tests, and troubleshooting.
- Apply basic prompt engineering and context engineering practices by providing the right technical background, constraints, schemas, examples, and acceptance criteria to improve output quality.
- Validate and verify AI-generated outputs before adoption, especially for correctness, security, and maintainability.
- Follow responsible AI usage expectations, including protection of sensitive data, credentials, and regulated information.

Qualifications Technical Experience

- 5-7+ years of experience in data engineering, data platforms, or related technical domains.
- Hands-on experience building and operating batch and/or streaming data pipelines.




- Experience working with Databricks-based platforms and lakehouse architectures.
- Robust familiarity with AWS data engineering services such as Amazon S3, AWS Glue, AWS Lambda, AWS Lake Formation, Amazon Kinesis, and AWS IAM.
- Working knowledge of Delta Lake, Apache Iceberg, and Databricks Unity Catalog (or equivalent governed lakehouse concepts).
- Solid understanding of data quality, schema management, lineage/metadata basics, access control, encryption, and operational support.

Leadership and Collaboration

- Ability to provide technical guidance without people management responsibility.
- Experience collaborating with global teams across time zones and working through ambiguous delivery problems.
- Strong communication skills and ability to translate technical requirements into implementable tasks.

AI Tooling (Required)

- Practical experience using AI-assisted engineering tools such as Claude, GitHub CoPilot, or LLM-based development features.
- Ability to use AI effectively with basic context engineering practices (inputs/constraints/examples/acceptance criteria).
- Demonstrated ability to validate AI-generated outputs and follow responsible, secure usage practices.

Preferred Qualifications

- Experience with enterprise onboarding frameworks, reusable ingestion patterns, and pipeline templates.
- Familiarity with data cataloging/metadata management and governed data product concepts.
- Knowledge of semantic modeling (e.g., semantic layers, business definitions, canonical models) is a plus.
- Exposure to enterprise data mastering/MDM workflows, golden record concepts, and trusted data distribution.
- Experience in regulated environments with strong governance, auditability, and security expectations.
- Databricks and/or AWS certifications (or equivalent practical experience) are a plus.

📌 Data Engineer, Data Platforms (Chennai)
🏢 Su0026P Global Market Intelligence
📍 Chennai