3Pillar is an AI transformation partner on a mission to help enterprises build
the AI-native products and intelligent agents that will define the next era of
business. With teams across North America, Europe, Latin America, and Asia, we
work with the most ambitious companies in financial services, healthcare, media,
and technology — helping them move faster, modernize boldly, and compete on
their own terms. Our HelixAI platform and Helix Pods delivery model put our
engineers at the center of real agentic transformation — doing work that is
open, portable, and built to last. We are building the future of enterprise AI

We are looking Lead Data Engineer to build, operate, and continuously improve
the
data pipelines, retrieval infrastructure, and ML/LLMOps foundations that power
our AI
initiatives. The resource will work on turning reference architectures and data
contracts
into robust, production-grade implementations that serve conversational AI
assistants,
dashboard copilots, autonomous agents, RAG applications, and predictive ML
models.

n

Key Responsibilities:




Data Pipeline Engineering : Build, test, and maintain production pipelines
(batch & real-time) on Snowflake, PySpark, Delta Lake, and Kafka.
Implement data quality checks, schema validation, and alerting at every pipeline
stage.
Migrate legacy ETL/DWH to cloud-native AWS/Azure architectures with measurable
latency and cost improvements.
Maintain CI/CD pipelines: automated testing, deployment, rollback, and IaC
(Terraform, GitHub Actions).

RAG, Vector & Retrieval Infrastructure: Build end-to-end retrieval
infrastructure: document ingestion, embedding pipelines, vector store management
(Pinecone, FAISS, ChromaDB, OpenSearch), and hybrid retrieval layers.
Implement chunking, metadata filtering, and re ranking — tuning for precision,
recall, and latency.
Maintain data freshness and index consistency; instrument with context relevance
and faithfulness metrics.

Semantic Layer & Knowledge Infrastructure: Implem

📌 Lead Data Engineer with AI experience (India)
🏢 3Pillar
📍 India