Hiring for Client - KnackForge

Key Responsibilities

- Architect and implement enterprise-grade Lakehouse solutions using Databricks
- Design and deliver scalable batch and real-time data pipelines using Apache Spark (PySpark/SQL)
- Build ETL/ELT pipelines, incremental data loads, and metadata-driven ingestion frameworks
- Implement and optimize Databricks components: Delta Lake, Delta Live Tables, Autoloader, Structured Streaming, and Workflows
- Design large-scale data warehousing solutions with 3NF and dimensional modeling
- Establish data governance, security, and data quality frameworks, including Unity Catalog
- Lead ML lifecycle management using MLflow and drive AI use cases (RAG, AI/BI)
- Manage cloud-native deployments on Microsoft Azure and integrate with enterprise systems (e.g., ServiceNow)
- Drive CI/CD, DevOps practices, and performance optimization of Spark workloads
- Provide technical leadership, mentor teams, and ensure successful delivery




- Collaborate with stakeholders to translate business requirements into scalable solutions

Ideal Candidate

Ideal Candidate — Databricks Lead

- 10 years of total experience, with 5 years specifically in Databricks
- Architecture-level expertise — must understand end-to-end design, not just engineering execution
- Hands-on with Delta Lake, Delta Live Tables, Spark (PySpark/SQL), and Azure Data Factory
- Prior experience working on Azure or Azure DevOps environments
- Has handled US/UK or international client-facing projects
- Comfortable managing and leading a team
- Basic understanding of AI/ML tools and services is a plus

Perks, Perks & Culture
- The company provides free AWS and Azure certification training
- Group medical insurance of ₹5 lakhs is included in the benefits package, along with meal allowances

📌 Data Engineer (Databricks, Python) (Chennai)
🏢 TalentXO
📍 Chennai