15 Nov
|
mckinsol consulting
|
Mumbai
15 Nov
mckinsol consulting
Mumbai
Key Roles & Responsibilities
1. End-to-End Solution Architecture
- Own solution architecture and design from source ingestion through Bronze, Silver and Gold layers to final
consumption.
- Work with business and technical stakeholders to understand functional and non-functional requirements.
- Translate business requirements into scalable data architectures, implementation patterns and engineering
designs.
- Determine the most appropriate architecture and technology based on performance, scalability, security, cost,
maintainability and operational complexity.
- Evaluate alternative designs and determine the appropriate processing and orchestration approach across
AWS, Snowflake, Apache Airflow and other approved platform components.
- Ensure solutions support current reporting and analytics requirements while also making enterprise data ready
for scalable downstream AI consumption where required.
- Define reusable architecture patterns and avoid isolated solutions where common platform capabilities can be
established.
1. Bronze, Silver & Gold Layer Ownership
- Define architecture, design principles and standards for Bronze, Silver and Gold layers.
- Determine what data should be captured, standardized, cleansed, transformed and modeled at each layer.
- Define appropriate patterns for full load, incremental load, CDC, MERGE, SCD, historization, deduplication,
reconciliation and retention.
- Determine the appropriate consumption layer based on business and technical requirements rather than
creating unnecessary Gold datasets.
- Ensure Silver and Gold datasets contain the appropriate business context, relationships, quality and history
required by downstream consumers. DATA SOLUTION ARCHITECT |
Confidential • 2
- Ensure datasets required for AI use cases are appropriately structured, standardized, contextualized, modeled
and governed before being made available for downstream AI consumption.
- Design the layers to maximize reuse across reporting, analytics and AI requirements, minimizing duplicate
processing and datasets.
1. Data Modeling
- Own conceptual, logical and physical data modeling for enterprise data solutions.
- Define the appropriate dimensional, relational and analytical modeling approach for each requirement.
- Establish standards for facts, dimensions, data grain, business keys, surrogate keys, relationships, common
dimensions and historical tracking.
- Define appropriate SCD and historization approaches based on business requirements.
- Ensure consistent business definitions and reusable models across different solutions.
- Design models capable of serving reporting, analytics and AI consumption requirements without unnecessary
duplication.
- Ensure consumption-ready datasets contain the necessary business context and relationships for reliable
downstream usage.
- Review and approve data models produced by implementation teams before development and production
deployment.
1. Engineering & Production Standards
Ensure every solution appropriately addresses: Data quality and validation Schema-change management
Source-to-target reconciliation Metadata and data lineage
Record-count reconciliation Scheduling and dependency management
Duplicate and rejected-record handling Security and role-based access
Error and exception management SLA/SLO requirements
Retry, restart and reprocessing mechanisms Performance and scalability
Logging and auditability Automated testing
Pipeline monitoring and alerting CI/CD and environment promotion
Failure email/notification mechanisms Production support and operational documentation The Solution Architect will ensure these requirements are incorporated into the solution design from the beginning rather than being addressed after development.
1. Technology Selection & Architecture Optimization
- Evaluate and recommend the most appropriate technology and implementation approach for each
requirement.
- Have strong architectural knowledge of Snowflake capabilities, including Snowpipe, Streams, Tasks, Dynamic
Tables, stored procedures and Iceberg.
- Have strong knowledge of relevant AWS services, including Amazon S3, AWS Glue, Glue Data Catalog,
Lambda, IAM and Amazon MWAA/Apache Airflow.
- Determine where ingestion, storage, transformation and orchestration should occur across AWS, Snowflake
and associated technologies based on architectural suitability.
- Consider downstream consumption through Snowflake Cortex and Azure-based data/AI capabilities when
designing datasets expected to support AI use cases.
- Ensure these datasets are structured and modeled appropriately for scalable downstream AI consumption.
DATA SOLUTION ARCHITECT |
Confidential • 3
- Identify inefficient, overly complex or costly implementations and recommend improvements.
- Continuously optimize performance, compute utilization, data movement, storage, scalability and cost.
- Reduce unnecessary duplication and establish reusable platform capabilities wherever appropriate.
- Periodically review existing architecture to identify opportunities for modernization, simplification and
optimization.
1. Architecture Governance & Design Review
- Act as the technical design authority for enterprise data solutions.
- Conduct architecture and solution-design reviews before implementation.
- Review designs created by engineering teams, implementation partners and other technology teams.
- Challenge solutions where scalability, maintainability, security, reliability, performance, cost or operational
readiness have not been sufficiently addressed.
- Ensure designs comply with established architectural, data-modeling and engineering standards.
- Ensure major architecture decisions and deviations are appropriately documented.
- Maintain Architecture Decision Records for significant technical decisions.
- Provide architecture and design approval before major solutions move into production.
1. Stakeholder & Engineering Collaboration
- Conduct requirement and solutioning workshops with business and technology stakeholders.
- Understand how the data will ultimately be consumed before defining the underlying architecture.
- Work closely with Data Engineers throughout implementation to ensure approved designs are implemented
correctly.
- Collaborate with business teams, application/source-system teams, reporting and analytics teams, AI teams,
security, infrastructure and implementation partners.
- Ensure downstream requirements are understood early enough to influence ingestion, transformation and
modeling decisions.
- Review implementations against the approved design and identify deviations or gaps.
- Communicate architecture decisions clearly to both technical and non-technical stakeholders.
Required Skills & Experience
- 812+ years of experience in Data Engineering, Data Warehousing, Data Architecture or related areas.
- Demonstrated experience owning end-to-end enterprise data solution architecture.
- Strong architectural and hands-on knowledge of Snowflake and AWS.
- Strong understanding of Bronze/Silver/Gold Medallion Architecture.
- Advanced expertise in data modeling, including dimensional modeling, facts/dimensions, SCD, CDC and
historization.
- Strong SQL skills and good working knowledge of Python.
- Robust understanding of ETL/ELT architecture, transformation frameworks and SQL-based data-processing
patterns.
- Experience with Apache Airflow / Amazon MWAA or equivalent orchestration platforms.
- Strong understanding of data quality, reconciliation, error handling, observability, metadata and lineage.
- Understanding of designing governed and consumption-ready datasets for downstream reporting, analytics
and AI workloads.
- Experience with enterprise security, RBAC, automated testing, CI/CD and production deployment controls.
- Strong understanding of cloud performance and cost optimization.
DATA SOLUTION ARCHITECT |
Confidential • 4
- Ability to evaluate multiple architectural alternatives and select the appropriate solution based on business
and technical requirements.
- Strong stakeholder management and business-to-technology translation capabilities.
📌 Data Solution Architect (Mumbai)
🏢 mckinsol consulting
📍 Mumbai