Job Description Senior Databricks Architect

Position

Senior Databricks Architect

Location

Offshore Bangalore

Domain

Retail / Grocery

Experience

10+ years overall IT experience, with 5+ years in Databricks / Cloud Data Engineering / Data Architecture

Role Overview

We are looking for an experienced Senior Databricks Architect to lead the architecture and technical execution of a large-scale Talend-to-Databricks migration for a leading Retail/Grocery client.

The architect will be responsible for designing the target Databricks Lakehouse architecture , defining migration patterns, guiding the data engineering team, and ensuring successful modernization of existing Talend-based ETL/ELT workloads onto Databricks.

The role requires strong hands-on expertise in Databricks, Apache Spark, SQL, cloud data platforms, data architecture, ETL modernization, and migration strategy , along with a good understanding of Retail/Grocery business processes and data domains .

Key Responsibilities

Architecture Solution Design

- Lead the end-to-end architecture for migration of Talend ETL/ELT workloads to Databricks.

- Define the target-state Databricks Lakehouse architecture , including data ingestion, transformation, storage, orchestration, governance, and consumption layers.

- Design scalable, secure, highly available, and cost-optimized data solutions.

- Define architecture standards, design patterns, coding standards, and best practices for Databricks development.

- Establish migration frameworks and reusable patterns for converting Talend jobs into Databricks/Spark-based pipelines.

- Evaluate existing Talend jobs and determine appropriate migration strategies, including re-platforming, re-engineering,



consolidation, or retirement.

- Provide technical leadership for complex data pipelines and integration scenarios.

Talend to Databricks Migration

- Analyze existing Talend workflows, jobs, mappings, dependencies, schedules, and data transformations.

- Develop migration strategies for batch and incremental data processing workloads.

- Translate Talend transformations and business rules into PySpark /SQL/Databricks implementations.

- Identify opportunities to simplify and optimize legacy ETL processes during migration.

- Define data reconciliation and validation strategies to ensure parity between Talend and Databricks outputs.

- Establish migration sequencing based on business criticality, dependencies, complexity, and risk.

- Support migration of high-volume and business-critical data pipelines.

Databricks Data Engineering

- Design and implement solutions using Databricks, Apache Spark, Delta Lake, PySpark , and SQL .

- Design Medallion Architecture (Bronze, Silver, Gold) and appropriate data processing patterns.

- Implement incremental processing, CDC, SCD Type 1/Type 2, data quality, error handling, and audit frameworks.

- Optimize Spark workloads, Delta tables, SQL queries, partitioning, clustering, and job execution.

- Leverage Delta Live Tables / Lakeflow capabilities ,



Workflows, Unity Catalog , and other Databricks platform services where appropriate.

- Design data pipelines integrating structured and semi-structured data from databases, files, APIs, and enterprise applications.

- Establish monitoring, logging, alerting, operational support, and performance-management patterns.

Cloud Integration

- Robust experience with at least one major cloud platform, preferably Azure or AWS .

- Design integration between Databricks and cloud storage, databases, messaging platforms, APIs, and enterprise applications.

- Experience with technologies such as ADLS/S3, Azure Data Factory, Kafka, REST APIs, relational databases, and cloud-native services is highly desirable.

- Define secure connectivity, networking, secrets management, and access-control patterns.

Retail / Grocery Domain

- Work closely with business and technology stakeholders to understand Retail/Grocery data requirements.

- Experience with retail data domains such as:

- Product / Item

- Store / Location

- Customer / Loyalty

- Sales / Transactions

- Pricing

- Promotions

- Inventory

- Supply Chain

- Vendors / Suppliers

- Orders

- Forecasting

- Merchandising

- Understand retail-specific challenges such as high-volume transaction processing, store-level data, product hierarchies, promotions, pricing, inventory movements, and customer analytics.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 SENIOR DATA ARCHITECT - Databricks (Bengaluru)
🏢 Happiest Minds Technologies
📍 Bengaluru