Databricks Forward Deployed Engineer (FDE)

Function: Data & AI Engineering

No of Roles- 20 roles

Senior G7- 5-8 yrs

Contract duration- 1 yr

Location- Any Fractal Location

Mandatory skills- Python, react, azure, gen AI, CICD

Role Overview The Databricks Forward Deployed Engineer (FDE) is a senior client-facing engineer embedded with enterprise clients to design, build, and optimise production-grade data platforms on Databricks. The role requires deep Spark and Databricks technical expertise combined with strong client communication skills — the FDE leads technical engagements end-to-end, makes defensible architecture decisions, and delivers measurable platform outcomes.

Key Responsibilities

Client Engagement & Solution Design

- Lead technical discovery sessions to understand client data platform challenges and objectives

- Design end-to-end Lakehouse architectures — Medallion (Bronze/Silver/Gold), streaming, batch, and hybrid patterns

- Present architecture recommendations and trade-off analyses to technical and executive stakeholders

- Produce solution architecture documents, data flow diagrams, and design decision records

Platform Engineering

- Build production data pipelines using PySpark, Spark SQL, and Delta Lake on Databricks

- Implement ingestion patterns using AutoLoader, Kafka, and event-driven architectures

- Implement SCD Type 1/2/3 and CDC patterns for dimension and fact table management

- Optimise Delta Lake tables — OPTIMIZE, VACUUM, Liquid Clustering, Z-Ordering, and Deletion Vectors

- Right-size and configure Databricks clusters — Job vs All-Purpose, Driver/Worker ratios, auto-scaling, spot instances

- Configure Unity Catalog — three-level namespace, data lineage, access controls, and workspace federation

Performance & Production Engineering

- Diagnose performance bottlenecks using Spark UI — stage analysis, shuffle, data skew, and OOM resolution

- Implement CI/CD pipelines using Databricks Asset Bundles (DABs), GitHub Actions, or Azure DevOps

- Define data quality checks, SLA monitoring,



and production alerting frameworks

- Quantify performance improvements and cost savings in business terms

GenAI & Advanced Analytics (Basic)

- Support GenAI solution design — RAG pipelines, vector store integration, and LLM evaluation on Databricks

- Leverage Mosaic AI and Databricks Genie for ML and analytics workloads

Required Qualifications

Experience

- 5+ years of data engineering experience in production environments

- 2+ years of hands-on Databricks or Apache Spark platform delivery at scale

- Prior client-facing, consulting, or embedded engineering experience

Technical Skills

- Deep PySpark and Spark SQL proficiency — execution model, stage/shuffle/task lifecycle, performance tuning

- Hands-on Delta Lake expertise — transaction log, ACID, time travel, schema evolution, compaction

- Strong Databricks cluster architecture knowledge — AQE, dynamic allocation, pool configuration

- Experience designing and delivering Medallion Architecture for enterprise Lakehouse platforms

- AutoLoader proficiency — incremental ingestion, schema inference and evolution, duplicate handling

- Strong SQL — window functions, CTEs, analytical queries, query optimisation

- CI/CD hands-on — DABs, notebooks-as-code, multi-environment deployment pipelines

- Unity Catalog or equivalent data governance and access control frameworks

Communication & Soft Skills

- Ability to present complex technical decisions clearly to non-technical stakeholders

- Structured problem-solving — can diagnose unfamiliar issues and arrive at solutions under ambiguity

- Ownership mindset — proactively identifies delivery risks and escalates effectively

Preferred Qualifications

- Databricks Certified Data Engineer Associate or Skilled certification

- Experience with Structured Streaming, Delta Live Tables, or Kafka streaming architectures

- GenAI / LLM exposure — RAG, vector databases, embedding pipelines, hallucination mitigation

- Cloud platform experience — Azure (preferred), AWS, or GCP

- Prior experience within a DPP, GSI, or consulting delivery model

📌 Databricks Forward Deployed Engineer (India)
🏢 Fractal
📍 India