Designation : Assistant Manager - Data Engineering

Level : L3

Location : Chennai

Experience : 5 to 8 Years

Job Role :

Build and maintain scalable data pipelines and data warehouse solutions using Python, PySpark, SQL, Databricks, and Airflow.
Drive data quality, workflow orchestration, performance optimization, and Linux-based data engineering, while collaborating with cross-functional teams to deliver reliable analytics solutions.

Responsibilities :

Data Pipeline Development:

- Design, develop, and maintain scalable data pipelines using Python, PySpark, and SQL to ingest, process, and transform data from various sources.

- Implement data quality checks and monitoring to ensure the reliability and integrity of data pipelines.

Data Warehouse Management:

- Build and optimize data warehouse solutions for effective storage and retrieval of large volumes of structured and unstructured data.

- Collaborate with cross-functional teams to design and implement data models for analytical purposes.

Databricks Expertise:

- Utilize Databricks platform for data engineering tasks, including data ingestion, ETL processing, and data analysis.

- Optimize Databricks jobs and clusters for performance and cost efficiency.

Airflow Workflow Orchestration:

- Configure and manage Airflow workflows to automate data pipeline orchestration and scheduling.

- Implement error handling and monitoring within Airflow DAGs to ensure reliable execution of data workflows.

Linux Administration:





- Manage Linux-based servers and environments for data engineering tasks, ensuring security, reliability, and performance.

- Troubleshoot and resolve issues related to Linux infrastructure and server configurations.

Collaboration and Documentation:

- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.

- Document data engineering processes, workflows, and best practices for knowledge sharing and future reference.

Required skills :

- Bachelor's degree in Computer Science, Engineering, or related field.

- Proven experience as a Data Engineer or similar role, with a minimum of 6 to 8 years of relevant experience.

- Proficiency in Python, PySpark, SQL, and Databricks.

- Good to have familiarity with Claude Code or Github Copilot or Codex ,GEN AI - Databricks Genie, LLM , Prompt engineering

- Familiarity with Airflow for workflow orchestration.

- Experience with Linux administration and shell scripting.

- Strong understanding of data modelling, ETL processes, and data warehousing concepts.

- Excellent problem-solving skills and attention to detail.

- Strong communication and collaboration skills, with the ability to work effectively in a team environment.

Job Snapshot

Updated Date

23-09-2026

Job ID

J_5964

Location

Chennai, Tamil Nadu, India

Employee Type

Permanent

📌 Assistant Manager - Data Engineering (Chennai)
🏢 LATENTVIEW ANALYTICS
📍 Chennai