Job DescriptionKey Responsibilities:
NPipeline development — Design, build, and maintain robust data pipelines to ingest new datasets onto the firm's data platform as demand arises from the investment and analyst teams. Automation — Automate ingestion, transformation, and data-quality workflows to reduce manual effort and speed up onboarding of current sources.
NData platform — Structureand organize datasets so they can be integrated, joined, and queried effectively across the platform.
N
- AI enablement (roadmap)— Process large volumes of documents into formats suitable for AI consumption;
store and index contentin vector databases;
support retrieval workflows (RAG) that surface the right information reliably.
N
- Insight & reporting (roadmap) — Enable effective querying across multiple integrated datasets to combine information and support sensible, decision-ready reporting.
N
- Stakeholder collaboration — Work directly with the hiring manager and analysts to scope new requirements, prioritise the pipeline backlog, and respond quickly as new needs emerge.




N nKey Competencies:
N nRequired Skills
- 4–5+years of hands-on data engineering experience, ideally within financial services or data intensive environments.
N
- Strong experience building and orchestrating data pipelines with Apache Airflow.
- Solid working knowledge of Apache Spark for large-scale data processing.
N
- Strong Python and SQL skills, with a focus on clean, maintainable, production-quality code. Experience designing data models and organizing datasets for downstream integration and querying.
N
- Track record of automating data workflows and building for reliability, monitoring, and data quality.
N nNice to Have.
n n
- Exposure to AI/LLM engineering: document processing at scale, embeddings, and vector databases (e.G. pgvector, Pinecone, Weaviate, or similar).
N
- Experience building or supporting retrieval-augmented generation (RAG) workflows.
N
- Familiarity with cloud

📌 Technical Lead (Surat)
🏢 Acuity Analytics
📍 Surat