?

Senior Databricks

Architect

Position

Senior Databricks Architect

Location

Offshore ? Bangalore

Domain

Retail / Grocery

Experience

10+ years overall IT experience, with 5+ years in Databricks / Cloud Data Engineering / Data Architecture

Role Overview

We are looking for an experienced?Senior Databricks Architect?to lead the architecture and technical execution of a large-scale?Talend-to-Databricks migration?for a leading Retail/Grocery client.

The architect will be responsible for designing the target?Databricks Lakehouse architecture, defining migration patterns, guiding the data engineering team, and ensuring successful modernization of existing Talend-based ETL/ELT workloads onto Databricks.

The role requires strong hands-on expertise in?Databricks, Apache Spark, SQL, cloud data platforms, data architecture, ETL modernization, and migration strategy, along with a valuable understanding of?Retail/Grocery business processes and data domains.

Key Responsibilities

Architecture & Solution Design

- Lead the end-to-end architecture for migration of Talend ETL/ELT workloads to Databricks.
- Define the target-state?Databricks Lakehouse architecture, including data ingestion, transformation, storage, orchestration, governance, and consumption layers.
- Design scalable, secure, highly available, and cost-optimized data solutions.
- Define architecture standards, design patterns, coding standards, and best practices for Databricks development.
- Establish migration frameworks and reusable patterns for converting Talend jobs into Databricks/Spark-based pipelines.




- Evaluate existing Talend jobs and determine appropriate migration strategies, including re-platforming, re-engineering, consolidation, or retirement.
- Provide technical leadership for complex data pipelines and integration scenarios.

Talend to Databricks Migration

- Analyze existing Talend workflows, jobs, mappings, dependencies, schedules, and data transformations.
- Develop migration strategies for batch and incremental data processing workloads.
- Translate Talend transformations and business rules into?PySpark/SQL/Databricks?implementations.
- Identify opportunities to simplify and optimize legacy ETL processes during migration.
- Define data reconciliation and validation strategies to ensure parity between Talend and Databricks outputs.
- Establish migration sequencing based on business criticality, dependencies, complexity, and risk.
- Support migration of high-volume and business-critical data pipelines.

Databricks & Data Engineering

- Design and implement solutions using?Databricks, Apache Spark, Delta Lake, PySpark, and SQL.
- Design?Medallion Architecture (Bronze, Silver, Gold)?and appropriate data processing patterns.
- Implement incremental processing, CDC, SCD Type 1/Type 2, data quality, error handling, and audit frameworks.




- Optimize Spark workloads, Delta tables, SQL queries, partitioning, clustering, and job execution.
- Leverage?Delta Live Tables / Lakeflow capabilities, Workflows, Unity Catalog, and other Databricks platform services where appropriate.
- Design data pipelines integrating structured and semi-structured data from databases, files, APIs, and enterprise applications.
- Establish monitoring, logging, alerting, operational support, and performance-management patterns.

Cloud & Integration

- Strong experience with at least one major cloud platform, preferably?Azure or AWS.
- Design integration between Databricks and cloud storage, databases, messaging platforms, APIs, and enterprise applications.
- Experience with technologies such as?ADLS/S3, Azure Data Factory, Kafka, REST APIs, relational databases, and cloud-native services?is highly desirable.
- Define secure connectivity, networking, secrets management, and access-control patterns.

Retail / Grocery Domain

- Work closely with business and technology stakeholders to understand Retail/Grocery data requirements.
- Experience with retail data domains such as:
- Product / Item
- Store / Location
- Customer / Loyalty
- Sales / Transactions
- Pricing
- Promotions
- Inventory
- Supply Chain
- Vendors / Suppliers
- Orders
- Forecasting
- Merchandising
- Understand retail-specific challenges such as high-volume transaction processing, store-level data, product hierarchies, promotions, pricing, inventory movements, and customer analytics.

Databricks

📌 SENIOR DATA ARCHITECT - Databricks (Bengaluru)
🏢 Happiest Minds Technologies
📍 Bengaluru