Job Description

About Us

/n

AiLogic Neural Network Pvt Ltd is an AI-driven product company focused on building advanced language technology solutions, including machine translation, document intelligence, and large-scale NLP systems. We are looking for a highly motivated Data Engineer to join our growing AI team and contribute to the development of scalable data processing pipelines for NLP and LLM applications.

/n

Roles & Responsibilities

/n

/n
Design, develop, and maintain scalable data pipelines for processing large volumes of structured and unstructured data.
/n
Build document ingestion and processing workflows for PDFs, scanned documents, HTML pages, and other text sources.
/n
Implement OCR, PDF parsing, HTML parsing, and text extraction pipelines.
/n
Develop document chunking and preprocessing frameworks for NLP and LLM-based applications.
/n
Work with Hugging Face models and NLP libraries for text processing tasks.
/n
Create and optimize data transformation workflows using Python, Apache Spark, and Spark SQL.
/n




Develop and manage Vector Database pipelines for embedding storage and retrieval.
/n
Implement text normalization, sentence segmentation, deduplication, and data quality processes.
/n
Design and implement data masking, classification, and categorization solutions.
/n
Collaborate with AI/ML engineers to prepare datasets for model training and inference.
/n
Optimize large-scale data processing workflows for performance, scalability, and cost efficiency.
/n
Maintain CI/CD pipelines and follow software engineering best practices.
/n
Monitor, troubleshoot, and improve production data processing systems.
/n

/n

Mandatory Skills

/n

/n
Robust experience with Python programming.
/n
Hands-on experience in NLP concepts such as:
/n
Tokenization
/n
Text Processing
/n
Hugging Face Transformers
/n
Experience in:
/n
PDF Parsing
/n
OCR
/n
HTML Parsing
/n
Text Extraction
/n
Document Chunking
/n
Experience with

📌 Data Engineer Hyderabad
🏢 Ailogic Neural Network
📍 Hyderabad