07 Nov
|
Springml India Development Center Private
|
Hyderabad
07 Nov
Springml India Development Center Private
Hyderabad
Principal Data ArchitectWe are seeking a Principal Data Architect to define and drive enterprise data architecture, modernization, and governance strategies across complex multi-cloud (Google Cloud Platform and Microsoft Azure) and hybrid environments. This role will design scalable, secure, and high-performing data platforms while partnering with business and engineering leaders to translate strategic objectives into impactful data solutions.A key focus of this role will be spearheading scalable, agent-driven data migration pipelines to transition legacy on-premises data assets to the cloud, setting the technical direction for engineering teams, and actively managing the customer's technical backlog and delivery roadmap.Key Responsibilities : - Multi-Cloud Lakehouse Architecture : Lead the end-to-end architecture, design, and implementation of scalable Data Lakehouse solutions across Google Cloud Platform (BigQuery, GCS, BigLake, Dataplex) and Microsoft Azure (Azure Synapse Analytics, Microsoft Fabric, ADLS Gen2, Azure Databricks).- On-Prem to Cloud Migration & Agentic Pipelines : Architect and deploy automated, scalable, agent-driven data pipelines (leveraging GenAI agents and metadata-driven frameworks) to orchestrate, validate, and accelerate complex migrations from on-premises legacy warehouses/databases (Oracle, Teradata, SQL Server) to GCP and Azure.- Client Engagement & Advisory : Collaborate directly with customers to understand business goals, data challenges, and technical requirements; translate them into robust architectural blueprints, technical roadmaps, and actionable delivery plans.- Customer Technical Backlog Management : Own and actively manage the customer technical backlog - translating business epics into technical user stories, prioritizing technical debt, defining sprint milestones, and aligning backlog items with business ROI.- Modern Ingestion & Streaming : Design and implement data pipelines supporting both real-time and batch ingestion using modern orchestration (Cloud Composer, Azure Data Factory, Airflow) and streaming/messaging frameworks (Kafka, GCP Pub/Sub, Azure Event Hubs).- Governance & Lineage : Establish and enforce best practices for unified data cataloging, metadata management, automated lineage,
and data quality across multi-cloud and on-prem systems using Dataplex, Microsoft Purview, or open-source governance tools.- Security & Compliance : Define and implement robust data security, identity management (IAM / Azure Entra ID), encryption, row/column-level access control, and regulatory compliance standards across clouds.- Team Technical Leadership & Mentorship : Steer the overarching technical direction of engineering teams; establish architectural standards, mentor senior and mid-level engineers, conduct architecture review, and ensure engineering excellence.- Hands-on Delivery & Prototyping : Balance strategic architecture discussions with hands-on solutioning, architectural spikes, Proof-of-Concepts (POCs), and performance tuning of complex multi-cloud distributed workloads.- Cross-Functional Partnership : Partner with enterprise architects, delivery leads, DevOps engineers, and customer stakeholders to drive solution adoption, operational scalability, and long-term platform maintainability.Technical Experience & Tooling : - Cloud Platforms : Deep expertise in both Google Cloud Platform (GCP) and Microsoft Azure enterprise data ecosystems.- Processing Engines : Apache Spark, PySpark, Databricks, Azure Synapse / Fabric Spark, GCP Dataproc.- Modern Migration & AI Automation : Building automated, agentic, or metadata-driven data migration pipelines, automated schema conversion, and data validation agents for on-prem to cloud lift-and-shift/re-platforming.- Messaging & Streaming : Apache Kafka, GCP Pub/Sub, Azure Event Hubs.- Orchestration : Apache Airflow, Google Cloud Composer, Azure Data Factory (ADF).- Analytical Storage & Warehouses : GCP BigQuery, Azure Synapse, Microsoft Fabric, Delta Lake/Parquet.- NoSQL / Cache Storage : MongoDB, Redis, Bigtable, Azure Cosmos DB.- Containerization & Deployment : Kubernetes (GKE and AKS), Cloud Run,
Azure Container Apps.- API Frameworks : Python (FastAPI, Flask) for data service endpoints.- Databases & Querying : Advanced SQL validation and extraction from legacy/RDBMS systems (SQL Server, Oracle, PostgreSQL).- CI/CD & Version Control : Git, GitHub, GitLab CI/CD, Azure DevOps, and Infrastructure as Code (Terraform).Required Qualifications & Skills : - 15+ years of progressive experience in Software Engineering, Data Engineering, and Enterprise Data Platform architecture.- 5+ years architecting, migrating, and deploying production data platforms across GCP and/or Azure, with proven multi-cloud integration patterns.- Demonstrated track record leading on-premises/cloud to cloud modernization/migration initiatives, including data ingestion, transformation, automated data reconciliation, and cutover strategies.- Experience designing or integrating autonomous, agent-driven, or LLM-augmented data workflows (e.g., automated schema mapping, code conversion, data quality validation agents).- Proven hands-on experience designing and deploying Lakehouse platforms incorporating Medallion Architecture (Bronze/Silver/Gold) and Data Mesh / Data Product principles.- Deep knowledge of multi-cloud data governance, access management, and cataloging (Dataplex, Microsoft Purview, Unity Catalog).- Solid leadership capability in technical backlog management, agile scoping, sprint planning, and setting technical strategy for distributed teams.- Exceptional stakeholder communication skills : capability to articulate architectural trade-offs to C-level executives as smoothly as reviewing code with engineers.Preferred Qualifications : - Multi-cloud certifications : Google Cloud Professional Data Engineer / Cloud Architect and/or Microsoft Certified : Azure Data Engineer Associate / Azure Solutions Architect Expert.- Hands-on knowledge of Snowflake and Databricks.- Familiarity with AWS data services (Amazon S3, Redshift, Glue, EMR).- Experience with LangChain, LlamaIndex, or agentic framework integrations in data operations.Education : Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or equivalent practical experience. (ref:hirist.tech)
📌 Egen - Principal Data Architect (Hyderabad)
🏢 Springml India Development Center Private
📍 Hyderabad