08 Oct
|
LATENTVIEW ANALYTICS
|
India
08 Oct
LATENTVIEW ANALYTICS
India
Job Role :
Role Overview:
We are seeking a senior Data Engineer with extensive hands-on experience building enterprise data platforms on Databricks, alongside proven experience implementing Generative AI solutions using Databricks Genie. In this role, you will architect, scale, and maintain end-to-end data pipelines, enforce enterprise data governance using Unity Catalog, and deliver production-grade GenAI-powered data workflows for business stakeholders.
Key Responsibilities
Data Engineering & Architecture
• Architect, build, and optimize complex, scalable ETL/ELT data pipelines using PySpark, Databricks Workflows, Delta Lake, and Delta Live Tables (DLT).
• Implement robust data modeling, warehousing, and lakehouse patterns to support both analytical and operational use cases.
• Optimize Databricks clusters, query performance, and computational costs across high-volume enterprise datasets.
GenAI & Enterprise Delivery
• Design, implement, and deploy production-grade Databricks Genie enterprise data applications.
• Enable natural-language query interfaces over enterprise data lakes, integrating Genie with existing data assets, metadata, and business logic.
• Ensure seamless delivery of GenAI workloads, from prototype to production deployment, adhering to enterprise SLA, security, and accuracy standards.
Data Governance & Security
• Implement and manage fine-grained data access controls, row/column-level security, and audit policies using Unity Catalog.
• Maintain centralized data discovery, lineage tracking, and metadata management across multi-workspace environments.
Core Requirements
Experience & Education
• Data Engineering: 7+ years of experience in data engineering, data platform development, and big data architecture.
• Databricks Expertise: Solid proficiency in Delta Lake, PySpark/Scala, Delta Live Tables, Databricks Workflows, and performance tuning.
• GenAI Experience: 1–2 years of hands-on experience deploying Generative AI solutions, specifically de
📌 Data Engineer (India)
🏢 LATENTVIEW ANALYTICS
📍 India