Senior AI Data Engineer
Experience Level: 5 – 6 Years
Location: Pune, India
Company: OneLab Ventures

About the RoleWe are seeking an experienced and hands-on Senior AI Data Engineer with 4–6 years of core expertise in building robust data pipelines, scaling big data architectures, and driving enterprise-grade metadata and master data management strategy.In this role, you will architect, optimize, and maintain high-throughput data pipelines (batch & streaming), manage automated workflows, and implement metadata cataloging and Master Data Management (MDM) solutions to power next-generation AI/ML applications and advanced analytics.

Key Responsibilities
- Big Data & Pipeline Engineering: Design, build, and optimize scalable batch and real-time streaming ETL/ELT pipelines using Apache Spark, Kafka, and modern cloud platforms (Databricks, Snowflake).
- Hadoop & Distributed Processing: Manage and optimize distributed data storage and computation across Hadoop ecosystems and cloud object storage.
- Metadata & MDM Governance: Implement and maintain robust Metadata Management,



Data Catalogs, and Master Data Management (MDM) frameworks to ensure data lineage, compliance, and high data quality across the organization.
- Workflow Orchestration: Build, schedule, and maintain complex, resilient DAGs and workflow pipelines using Apache Airflow.
- AI & Advanced Analytics Support: Structure, clean, and optimize large-scale unstructured/structured datasets specifically tuned for AI/ML models and analytics consumers.
- Performance Tuning: Continuously optimize query performance, data partitioning, memory management, and compute costs across Databricks and Snowflake environments.

Key Qualifications
- Experience: 4–6 years of solid experience in Data Engineering and Big Data domain.
- Core Tech Stack: Deep proficiency in Apache Spark, Kafka, Hadoop, and Apache Airflow.
- Contemporary Data Stack: Hands-on expertise with Databricks (Delta Lake, PySpark) and Snowflake.
- Data Governance & Arc

📌 Senior Data Engineer (Pune)
🏢 Onelab Ventures
📍 Pune