Cognoverse is looking for a specialized Data Engineer to manage and optimize our high-volume text ingestion pipelines. Our platform processes massive amounts of legal transcripts, using AI to convert unstructured documents into structured, query able data. A significant portion of this pipeline is already built, but we need an expert to step in, troubleshoot edge cases (like irregular document formatting that breaks the parser), and make the overall ingestion process robust, automated, and highly scalable.

Key Responsibilities:

- Pipeline Optimization: Maintain, improve, and automate our existing data ingestion program that converts raw text transcripts and PDFs into structured database records.

- Troubleshooting & Formatting Fixes: Rapidly diagnose and fix parsing failures caused by new, unexpected, or messy document formats. Build resilient logic to handle formatting edge cases smoothly.

- OCR & Text Extraction: Apply OCR technologies and text parsing scripts to accurately extract data from various document types (including scanned images and complex PDFs).

- AI Model Integration: Leverage AI and Large Language Models (LLMs)



to intelligently parse unstructured text, identify key entities, and map them to structured JSON/database schemas.

- Data Structuring: Ensure extracted data is clean, validated, and efficiently stored in the database for downstream application use.

Requirements:

- 2+ years of hands-on experience in Data Engineering, specifically focused on processing large volumes of text data.

- Proven experience working with OCR tools, text parsing libraries (Regex, document parsers), and AI models (LLMs, transformers) for data extraction.

- Strong programming skills, preferably in Python, with a knack for debugging complex text-processing scripts.

- Experience converting unstructured text (TXT, PDF) into structured formats (JSON, SQL/NoSQL databases).

- A problem-solving mindset with the patience and skill to unravel tricky data formatting issues.

- Must be able to work full time from the Noida office during regular IST business hours.Role & responsibilities

📌 Data Engineer (Noida)
🏢 Cognoverse Ai
📍 Noida