About the job
Senior Data Architect - Databricks
What are we looking for real solver?
Solver? Absolutely. But not the usual kind. We're searching for the architects of the audacious & the pioneers of the possible. If you're the type to dismantle assumptions, re-engineer 'best practices,' and build solutions that make the future possible NOW, then you're speaking our language.
Your Responsibilities what you will wake up to solve.
You are not just a Solutions Architect; you are a futurifier of our data universe and the primary enabler of our AI ambitions. With a deep-seated passion for data engineering, you will architect and build the foundational Lakehouse infrastructure—on Databricks—that powers the customer's entire data intelligence ecosystem.
As the Directly Responsible Individual (DRI) for our enterprise-grade Databricks platforms, you own the outcome, end-to-end. You are the definitive solver for our customers' most complex data challenges, leveraging a powerful tech stack centered on the Databricks Lakehouse Platform (Apache Spark, Delta Lake, Unity Catalog, Databricks Workflows, MLflow) across major clouds (AWS, Azure, GCP). This is a hands-on-keys role where you won't just design solutions—you'll build them, break them, and perfect them.
- Solution Design & Pre-Sales Excellence: Partner with sales, engineering, and delivery teams to lead architecture workshops, solution design sessions, and technical roadmap discussions. Shape client-ready proposals and proofs-of-concept that position the Databricks Lakehouse as the platform of choice, and ensure successful, measurable project delivery.
- Design Core Data Engineering on Databricks: Master data modeling and medallion (Bronze/Silver/Gold) architecture, architecting high-performance ingestion and transformation pipelines with Apache Spark, Delta Lake, and Delta Live Tables while ensuring data quality and governance throughout the data lifecycle.
- Enable Cloud & AI: Design and implement Databricks solutions across major clouds (AWS, Azure, GCP), building foundational Lakehouse platforms—governed by Unity Catalog and powered by MLflow—that efficiently support advanced analytics, GenAI, and AI/ML initiatives.
- Optimize Performance & Cost: Continuously optimize Databricks architectures and implementations for performance, efficiency, and cost-effectiveness (Photon, cluster policies, and workload tuning) within the cloud environment.
- Bridge Business & Tech: Translate complex business requirements into clear technical designs and implementation plans, providing technical leadership and guidance to data engineering teams and executive stakeholders alike.
- Stay Ahead of the Curve:
Continuously research and evaluate new Databricks capabilities, architectural patterns (Lakehouse, Data Mesh, real-time streaming), and industry trends to keep our data platforms at the cutting edge.
Welcome to Searce The 'process-first', AI-native modern tech consultancy that's rewriting the rules. We don't do traditional. As an engineering-led consultancy, we are dedicated to relentlessly improving real business outcomes. Our solvers co-innovate with clients to futurify operations and make processes smarter, faster & better. We build alongside our clients—not for the vanity metrics, but for the transformation to embed lasting competitive advantage for our clients. The result? Modern business reinvention, built on math, tech, and clarity of purpose.
Functional Skills
- Enterprise Lakehouse Architecture Design: Expert ability to design holistic, scalable, and resilient Databricks Lakehouse architectures for complex enterprise environments.
- Cloud Data Platform Strategy: Proven capability to strategize, design, and implement cloud-native Databricks platforms across AWS, Azure, or GCP.
- Pre-Sales & Technical Storyteller: Crafts compelling, client-ready proposals, architectural decks, and technical demonstrations. Doesn't just present; shapes the strategic technical narrative behind every proposed Databricks solution.
- Advanced Data Modelling: Mastery in designing data models—including medallion architecture—for analytical, operational, and transactional use cases.
- Data Ingestion & Pipeline Orchestration: Strong expertise in designing and optimizing robust ingestion and transformation pipelines with Spark, Delta Live Tables, and Databricks Workflows.
- Governance & Security: Deep command of Unity Catalog for data governance, lineage, and role-based access control across the Lakehouse.
- Stakeholder Communication: Exceptional skills in articulating complex technical concepts and architectural decisions to both technical and non-technical (including executive-level) stakeholders.
- Performance & Cost Optimization: Adept at optimizing Databricks solutions for performance, efficiency, and cost within a cloud environment.
Tech Superpowers
- Databricks Lakehouse Mastery: You're a wizard with the Databricks Data Intelligence Platform—Apache Spark, Delta Lake, Unity Catalog, Databricks Workflows,
and Databricks SQL—with deep expertise deploying it across at least one major cloud (AWS, Azure, or GCP).
- Data Engineering Core: Highly skilled in designing, implementing, and managing data workflows and orchestration (Databricks Workflows, Airflow) and streaming (Spark Structured Streaming, Kafka). You're an authority on advanced data modeling, ETL/ELT patterns, ACID transactions, and data versioning.
- AI/ML Data Foundation: You instinctively design data pipelines and structures that efficiently feed and empower Machine Learning and Artificial Intelligence applications, with hands-on MLflow and a track record of enabling GenAI/LLM use cases.
- Programming for Data: You have strong command over key programming languages (Python, SQL; Scala a plus) for scripting, automation, and building data processing applications, plus CI/CD for data (Databricks Asset Bundles, Git) and infrastructure-as-code (Terraform).
Experience & Relevance
- Architectural Leadership (8+ Years): You bring extensive experience (8+ years) specifically in a Solutions Architect role focused on data engineering and platform building, including customer-facing consulting or qualified-services engagements.
- Databricks & Cloud Data Expertise: You have a proven track record of designing and implementing production-grade Databricks Lakehouse solutions leveraging major public cloud platforms (AWS, Azure, or GCP).
- Certified Databricks Expertise: Databricks Certified Data Engineer Professional certification is required, validating advanced, hands-on mastery of the Lakehouse Platform. Additional Databricks certifications (e.g., Machine Learning Professional or a Solutions Architect accreditation) are a strong plus.
- Data Warehousing & Lakehouse: Demonstrated hands-on experience in the end-to-end design, implementation, and optimization of modern lakehouses and comprehensive data platforms, including medallion architecture and Unity Catalog governance.
- Data Ingestion & Orchestration: Proven experience designing and implementing complex ingestion pipelines and workflow orchestration using Databricks Workflows and/or Airflow, and real-time streaming technologies like Kafka and Spark Structured Streaming.
- AI/ML Data Enablement: Experience building data foundations specifically geared towards supporting Machine Learning and Artificial Intelligence initiatives—including GenAI—with MLflow and MLOps practices.
Join the 'real solvers' ready to futurify?
If you are excited by the possibilities of what an AI-native, engineering-led, modern tech consultancy can do to futurify businesses, join and experience the 'Art of the possible'.
📌 Senior Data Architect - Databricks (India)
🏢 Searce
📍 India