This job is with S&P; Global, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly.

About the Role:

Grade Level (for internal use):

10

Key Responsibilities

Data Pipeline Transition and Platform Delivery

- Partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform.
- Provide hands-on engineering support to stabilize and optimize pipelines post-transition, focusing on performance, scalability, maintainability, and operational reliability.
- Implement and follow repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring.
- Support secure AWS deployment patterns and governed data access using AWS IAM, encryption, and platform guardrails.
- Work with lakehouse technologies such as Delta Lake, Apache Iceberg, and Databricks Unity Catalog, and support metadata-driven pipeline design.
- Assist with establishing and maintaining observability practices (logging, telemetry, alerts, dashboards) for production pipelines.

Data Onboarding and Asset-Agnostic Enablement

- Support asset-agnostic onboarding by applying standard ingestion and transformation frameworks to current datasets.
- Help offshore teams adopt common onboarding standards, delivery practices, and technical templates.
- Contribute to reusable technical assets such as reference implementations, onboarding templates, documentation, and operational runbooks.
- Support both batch and streaming onboarding patterns using AWS services (e.g., AWS Glue, AWS Lambda, Amazon Kinesis)



or event-driven integrations as appropriate.

Data Mastering Platform Integration

- Support integration of platform datasets with the enterprise data mastering platform.
- Help establish and/or maintain data quality controls, reconciliation, and metadata alignment for trusted mastered data.
- Support golden record / golden copy generation and maintenance activities for key financial datasets (as applicable to the program).

Technical Leadership and Engineering Excellence

- Provide technical guidance to engineering peers through code reviews, design reviews, and implementation planning.
- Contribute to engineering standards and delivery rigor across CI/CD, infrastructure automation, testing practices, and operational readiness.
- Help ensure documentation quality and operational discipline (runbooks, support procedures, incident learnings).
- Act as a technical bridge between offshore execution teams and global platform stakeholders to resolve delivery blockers and keep alignment.

AI-Assisted Engineering and Context Engineering

- Use AI tools (Claude, GitHub CoPilot, LLMs) to accelerate engineering tasks such as drafting code, refactoring, writing documentation, generating unit tests, and troubleshooting.




- Apply basic prompt engineering and context engineering practices by providing the right technical background, constraints, schemas, examples, and acceptance criteria to improve output quality.
- Validate and verify AI-generated outputs before adoption, especially for correctness, security, and maintainability.
- Follow responsible AI usage expectations, including protection of sensitive data, credentials, and regulated information.

Required Qualifications

Technical Experience

- 5-7+ years of experience in data engineering, data platforms, or related technical domains.
- Hands-on experience building and operating batch and/or streaming data pipelines.
- Experience working with Databricks-based platforms and lakehouse architectures.
- Strong familiarity with AWS data engineering services such as Amazon S3, AWS Glue, AWS Lambda, AWS Lake Formation, Amazon Kinesis, and AWS IAM.
- Working knowledge of Delta Lake, Apache Iceberg, and Databricks Unity Catalog (or equivalent governed lakehouse concepts).
- Solid understanding of data quality, schema management, lineage/metadata basics, access control, encryption, and operational support.

Leadership and Collaboration

- Ability to provide technical guidance without people management responsibility.
- Experience collaborating with global teams across time zones and working through ambiguous delivery problems.
- Strong communication skills and ability to translate technical requirements into implementable tasks.

AI Tooling (Required)

- Practical experience using AI-assisted engineerin

📌 Data Engineer, Data Platforms (India)
🏢 S&P Global
📍 India