Role & responsibilities

Senior Python Data Engineer AI & Machine Learning

Experience

68 Years

Job Summary

We are seeking an experienced Python Data Engineer who can design, build, and lead AI/ML data engineering solutions at an enterprise scale. The candidate will play a key role in enabling machine learning pipelines, AIdriven analytics, and data platforms, while also leading and mentoring a team of engineers.

Key Responsibilities

Data Engineering & AI/ML Enablement

- Design, develop, and maintain scalable data pipelines to support AI and Machine Learning use cases.
- Build efficient ETL/ELT workflows for structured and unstructured data.
- Prepare, transform, and optimize datasets for training, testing, and deploying ML models.
- Collaborate closely with data scientists to productionize ML and AI models.

Python Development

- Develop robust, scalable, and reusable Pythonbased data processing frameworks.
- Optimize data processing for performance, reliability, and scalability.
- Ensure high code quality through code reviews, testing, and best practices.

Platform & Architecture

- Design and implement cloudbased or onprem data architectures for AI/ML.
- Work with big data and distributed processing frameworks where applicable.
- Ensure data security, governance, and compliance across pipelines.

Leadership & Team Development

- Lead, mentor, and guide a team of data engineers.




- Provide technical direction, task planning, and delivery oversight.
- Participate in architecture reviews, design discussions, and technology evaluations.
- Act as a bridge between engineering, data science, and business teams.

Required Skills & Qualifications

Mandatory Skills

- 6–8 years of experience in data engineering with solid Python expertise.
- Proven experience supporting Artificial Intelligence and Machine Learning workloads.
- Strong handson experience with:
- Python (Pandas, NumPy, PySpark, or similar)
- SQL and data modeling
- ETL/ELT pipeline development

- Experience working with largescale datasets.
- Solid understanding of ML lifecycle and data requirements for AI models.
- Prior experience leading or mentoring engineers.

Technical Skills

- Python for data engineering
- SQL & relational databases
- Big data frameworks (Spark/PySpark – preferred)
- Cloud platforms (AWS/Azure/GCP – preferred)
- Workflow orchestration (Airflow or similar – preferred)
- REST APIs and data integrations
- CI/CD and basic DevOps practices

Nice to Have

- Experience with MLOps practices and tools.
- Exposure to Deep Learning or Generative AI workflows.
- Knowledge of data governance, lineage, and monitoring.
- Agile/Scrum delivery experience.

Preferred candidate profile

📌 Data Engineer (Tamil Nadu)
🏢 LanceSoft
📍 Tamil Nadu