Role Overview

Build scalable data pipelines and platforms with Python, PySpark, and cloud data services to support analytics, reporting, and AI/ML initiatives.

Key Responsibilities

- Design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data.

- Ingest data from structured and unstructured sources into cloud data lakes and warehouses.

- Build data models and warehouses for analytics and reporting, and tune SQL for performance.

- Implement data validation, cleansing, transformation, and quality checks.

- Work with data scientists, analysts, and application teams, following Git, CI/CD, testing, and monitoring practices.

Must Have Skills

- 5 - 8 years of data engineering, with solid Python programming.





- Hands-on PySpark and Apache Spark on large-scale distributed datasets.

- Advanced SQL (query optimization and performance tuning), plus data warehousing and dimensional modeling.

- Strong experience with Generative AI and Large Language Models (LLMs).

- AWS, Azure, or GCP data services, with Git, CI/CD, and Agile/Scrum.

Nice to Have

- Apache Airflow or similar orchestration

- Kafka and real-time streaming

- Databricks, Delta Lake, Apache Iceberg, or Hudi

- Docker, Kubernetes, and Terraform

- Data governance and metadata management

📌 Data Engineer (PySpark & Generative AI) (Chennai)
🏢 Tenth Planet Technologies
📍 Chennai