Job Description

"Experience Data Modernization ? migration of an on premises SQL data warehouse to a target state Data Lake on Google Cloud (GCP), enabling metrics & reporting, advanced analytics, and GenAI use cases (natural language querying, accelerated summarization, cross domain trend analysis) leveraging PySpark based processing, cloud native DevOps CI/CD pipelines, and containerized deployments on OpenShift (OCP) to deliver scalable, secure, and high performance data solutions. 14+ years of overall IT experience, with strong experience in Data Architecture, Big Data Engineering, Cloud Data Platforms, ETL/ELT Frameworks, Data Lake Architecture, and Enterprise Data Modernization programs. Location & Work Mode Location: India, Bengaluru Work Mode: 3 days a week Work From Client Office About Program/Project The IAM Data Modernization project involves migrating an on-premises SQL data warehouse to a target state Data Lake in GCP cloud environment. Key highlights include: ? Integration Scope: 30+ source system data ingestions and multiple downstream integrations ? Capabilities: Metrics, reporting, and Gen AI use cases with natural language querying, advanced pattern/trend analysis, faster summarizations, and cross-domain metric monitoring ? Benefits: ? Scalability and access to advanced cloud functionality ? Highly available and performant semantic layer with historical data support ? Unified data strategy for executive reporting, analytics, and Gen AI across cyber domains This modernization establishes a single source of truth for enterprise-wide data-driven decision-making. The IAM Data Modernization program is focused on migrating and modernizing enterprise IAM data platforms from on-premises systems into a scalable, metadata-driven modern data architecture.



The current phase involves building a generic metadata-driven ETL framework to ingest data from multiple source systems into OCP / S3, followed by future integration with GCP / BigQuery for advanced analytics, reporting, and GenAI use cases. Key Responsibilities ? Own the end-to-end data architecture for the IAM Data Modernization program across OCP, S3, PostgreSQL metadata repository, and GCP / BigQuery future state architecture. ? Define and guide implementation of a metadata-driven generic ETL framework supporting multiple source types including databases, files, APIs, MongoDB, Splunk, VDS, and Active Directory. ? Architect reusable metadata models using MD_ static metadata tables and OPS_ operational/runtime tables for source configuration, pipeline definition, run tracking, audit logging, checkpointing, schema drift, and notifications. ? Provide architecture direction for MVC-based orchestration, including repository layer, service layer, wrapper, controller, execution plan models, and standardized request/response contracts. ? Design scalable ingestion patterns for full load, incremental load, watermark-based processing, CDC-style patterns, batch ingestion, API ingestion, and file-based ingestion. ? Guide design of S3 data lake zones including Landing, Raw, Archive, and Error zones, with clear promotion, archival, replay, and error-handling patterns. ? Define data quality, schema validation, schema drift detection, CLOB validation, checkpoint restartability, auditability, and operational resilience standards.



? Provide technical leadership for GCP / BigQuery architecture, including S3 to BigQuery integration, schema mapping, dataset design, partitioning, clustering, performance optimization, and secure connectivity. ? Review and guide CI/CD automation using GitHub Actions, JFrog, Harness, Liquibase, Helm, OCP deployments, and environment-specific configuration management. ? Provide technical guidance to team on Python, PySpark, SQL, BigQuery, PostgreSQL, data modeling, performance tuning, and production-ready framework design. ? Support UAT and production readiness by defining validation scenarios, reconciliation strategy, performance testing approach, security validation, runbooks, and sign-off criteria. ? Partner with client stakeholders to finalize architecture decisions, design documents, Confluence documentation, implementation standards, and technical roadmaps. ? Ensure the solution is secure, scalable, maintainable, metadata-driven, auditable, and extensible for future Splunk and Grafana observability integrations. Required Skills Big Data & Processing ? Hands on experience with PySpark for ETL/ELT, data transformation, and performance optimization ? Solid understanding of distributed data processing concepts Data & Cloud Architecture ? Strong experience designing data platforms on Google Cloud Platform (GCP) ? Experience with Data Lakes, data warehousing, and large scale migration programs Data Lake Architecture & Storage ? Proven experience designing and implementing data lake architectures (e.g., Bronze/Silver/Gold or layered models). ? Solid knowledge of Cloud Storage (GCS) design, including bucket layout, naming conventions, lifecycle polic

1

Open Positions

Java,Security

Skills Required

BENGALURU

Location

Java,Security

Desirable Skills

184066

Job Code

📌 GCP Data Architect (Bengaluru)
🏢 Persistent
📍 Bengaluru