Senior Technical Specialist

Gautam Buddha Nagar, Uttar Pradesh

Job Summary

We are looking for a Databricks expert with strong experience in platform setup, architecture, and AI/ML capabilities across Azure and/or GCP . The ideal candidate should bring hands-on expertise in building Databricks-based AI solutions , not just traditional data pipelines.

Key Responsibilities

- Design and implement Databricks architecture on Azure and/or GCP (workspace, clusters, networking, security).
- Lead platform setup, governance, and optimization (Unity Catalog, cost control, access management).
- Build and deploy AI/ML solutions using Databricks (MLflow, Feature Store, Model Serving).
- Develop GenAI / LLM solutions (RAG pipelines, embeddings, vector search).
- Integrate LLMs (Azure OpenAI / Vertex AI / other APIs) within Databricks.
- Deliver scalable Lakehouse architecture solutions .
- Collaborate with engineering and business teams for end-to-end AI use cases .
- Ensure performance tuning, reliability, and security of workloads

Skill Requirements





- Strong experience in Databricks setup & architecture (must-have) .
- Hands-on with Databricks AI stack :
- MLflow, AutoML
- Feature Store
- Model Serving
- RAG / vector search / embeddings

- Experience with PySpark, Spark optimization, Delta Lake .
- Solid exposure to cloud platforms :

- Azure Databricks + Azure services
- GCP Databricks + GCP ecosystem (BigQuery, GCS, Vertex AI)

- Experience integrating LLMs / GenAI solutions .
- Knowledge of Unity Catalog, governance, and security .

Other Requirements

- Experience in GenAI applications on Databricks
- Familiarity with LangChain / vector databases
- Exposure to real-time/streaming pipelines
- CI/CD and DevOps practices

Ideal Candidate

- Strong in Databricks platform + AI (not just data engineering)
- Hands-on in cloud setup (Azure/GCP) and AI delivery
- Able to design end-to-end AI solutions

📌 Senior Technical Specialist (Gautam Buddh Nagar)
🏢 HCLTech
📍 Gautam Buddh Nagar