We are seeking an experienced Senior Data Engineer with deep expertise in Data Engineering, AWS Cloud, and Enterprise GenAI platforms. The ideal candidate will have extensive experience designing and implementing scalable data platforms and Retrieval-Augmented Generation (RAG) solutions while working in highly regulated environments.

This role requires strong hands-on experience in Python, SQL, distributed data systems, AWS services, and enterprise-grade GenAI architectures with governed data access.

Key Responsibilities

- Design, build, and maintain scalable enterprise data platforms and data pipelines.
- Develop robust ETL/ELT pipelines for structured and unstructured data.
- Build secure, production-grade Retrieval-Augmented Generation (RAG) applications.
- Design document ingestion, chunking, embedding, indexing, and retrieval pipelines.
- Implement governed data access, metadata management, and security controls for GenAI applications.
- Develop scalable data orchestration workflows using Airflow or AWS Step Functions.
- Optimize data models and distributed processing frameworks for performance and scalability.
- Collaborate with AI, Data Science, Platform Engineering, Security, and Business teams to deliver enterprise AI solutions.
- Ensure compliance with security, governance, and regulatory requirements.

Required Skills

Core Technical Skills

- 10-15 years of Data Engineering / Platform Engineering experience.
- 3+ years of hands-on experience with GenAI, LLMs,



and Retrieval-Augmented Generation (RAG).
- Robust programming skills in Python and SQL.
- Expertise in data modelling, ETL/ELT, and distributed data systems.
- Experience designing scalable enterprise data architectures.

AWS Expertise

Hands-on experience with:

- Amazon S3
- AWS Glue
- AWS Lambda
- Amazon EMR
- Amazon Redshift
- Amazon Kinesis
- AWS IAM

Data Orchestration

Experience with one or more:

- Apache Airflow
- AWS Step Functions

GenAI / RAG

Hands-on experience with:

- Enterprise RAG architecture
- LLM application development
- Embedding models
- Prompt engineering
- Vector search and semantic retrieval
- Document processing and indexing
- Secure and governed enterprise data access

Vector Databases

Experience with one or more:

- OpenSearch Vector Engine
- Pinecone
- FAISS
- Chroma

Preferred Skills

- LangChain
- LlamaIndex
- Amazon Bedrock
- OpenAI APIs
- Docker
- Kubernetes
- Terraform or CloudFormation
- CI/CD pipelines
- Data governance and security best practices

Domain Experience

Candidates with experience in Banking, Financial Services, Insurance, or other highly regulated industries will be preferred.

Nice to Have

- Knowledge of AI governance and Responsible AI principles.
- Experience with enterprise-scale data platforms.
- Exposure to MLOps and LLMOps practices.
- Strong stakeholder communication and solution design skills.

📌 Software Engineer (Tamil Nadu)
🏢 Diverse Lynx
📍 Tamil Nadu