19 Nov
|
Lexicon Infotech
|
Chennai
19 Nov
Lexicon Infotech
Chennai
Position Overview
- Job Title: Lead Data Engineer
- Experience Required: 8+ Yrs
- Location: Chennai
- Payroll Partner: Lexicon Infotech
- Project Client: BU Soft Tech
We are seeking an experienced Senior Data Engineer to design, build, and maintain scalable data pipelines and distributed data processing solutions.
Mandatory / Primary Skills
- Cloud Platform: AWS or Azure
- Python
- PySpark
- SQL
Experience
- 5 to 8 years of hands-on experience in Data Engineering and building scalable data solutions.
- Robust hands-on experience with cloud-based data engineering platforms. AWS experience is preferred, but candidates with Azure experience are also welcome.
Technical Skills
- Strong hands-on expertise in Python and PySpark for large-scale data processing.
- Advanced proficiency in SQL, including complex queries, query optimization, and data analysis.
- Experience with AWS data engineering services is preferred. Relevant Azure data engineering experience is also acceptable.
- Robust understanding of Apache Spark and distributed data processing systems.
- Experience designing, developing, and maintaining scalable, high-performance ETL/ELT data pipelines.
- Experience in data ingestion, transformation, integration, and processing from multiple data sources.
- Experience working with structured, semi-structured, and unstructured data, including JSON, logs, PDFs, and other file formats.
- Strong knowledge of relational and non-relational databases.
- Experience in data quality checks, validation, performance tuning,
and optimization.
Secondary Skills – Generative AI (GenAI)
- Exposure to Generative AI (GenAI) and LLM-based applications.
- Understanding of RAG (Retrieval-Augmented Generation) and vector databases.
- Experience integrating LLMs with data pipelines and cloud-based data platforms.
- Familiarity with frameworks such as LangChain or LlamaIndex.
- Understanding of data preparation, ingestion, and transformation for GenAI applications.
- Exposure to embedding generation, document processing, and vector search.
- Familiarity with AI/ML workflows and GenAI use cases in data engineering.
GenAI experience is an added advantage and is not mandatory. Analytical & Problem-Solving Skills
- Ability to perform in-depth data analysis and troubleshoot complex data-related issues.
- Proven experience building scalable data solutions to address complex business requirements.
- Strong problem-solving and analytical skills.
Soft Skills
- Strong verbal and written communication skills.
- Good organizational and stakeholder-management skills.
- Proactive self-starter with the ability to work independently and take ownership.
Technical Agility
- Ability to adapt quickly to new technologies and data platforms.
- Willingness to stay updated with emerging trends and best practices in Data Engineering, cloud platforms, Big Data technologies, and Generative AI.
Pay: ₹2,511,000.00 - ₹3,000,000.00 per year
Work Location: In person
📌 Data Engineer (Chennai)
🏢 Lexicon Infotech
📍 Chennai