Job Title: Data Engineer

Experience: 6 - 9 Years

Category: Software Development / Engineering

Main Location: Chennai

Employment Type: Full Time

Educational Qualification: Bachelor's degree in Computer Science or a related field, or higher, with a minimum of 6 years of relevant experience.

Position Description

We are seeking an experienced and skilled Staff / Data Engineer to join our growing data team. In this pivotal role, you will be instrumental in designing, building, and validating end-to-end data solutions while driving data democratization across the organization. You will work closely with business users and engineering teams to understand requirements and translate them into scalable technical solutions.

Knowledge of data modeling and data standards is desirable as a secondary skill. The ideal candidate should be able to understand data structures, business definitions, and standardization requirements to support consistent, reliable, and high-quality data processing. Additionally, you will be expected to take ownership of data platforms and processes, ensuring that data is accessible, trustworthy, and effectively utilized by end users.

Key Responsibilities

- Design & Development: Independently design, develop, and implement scalable, high-performance data pipelines for batch and real-time processing using Big SQL and PySpark. Orchestrate and manage data workflows, tasks, and batch jobs using Apache Airflow and Control-M
- Data Modeling: Should be able to visualize, build, and stitch physical dimensional models based on functional requirements within the automotive industry.
- Infrastructure Management: Should have advanced knowledge of creating CI/CD pipelines using Tekton,Github Actions and Terraform.
- Performance Optimization: Identify and resolve performance bottlenecks in data pipelines and queries by using query optimization techniques, splitting large processes into smaller chunks without impacting functionality.




- Data Governance & Quality: Implement and enforce data governance best practices, ensuring data quality, integrity, security, and compliance.
- Collaboration: Work closely with product managers and plant engineering teams to understand data requirements and translate them into technical solutions.
- Automation: Automate data extraction, transformation, and loading (ETL/ELT) processes by building reusable components.
- Observability : Should have experience in production batch process observability and the ability to articulate incidents and work on their permanent resolution.

Required Qualifications

- Bachelor's or master's degree in computer science, Engineering, Data Science, or a related field.
- 5+ years of professional experience as a Data Engineer, with a solid track record of designing and implementing complex data solutions.
- Expert proficiency in Advanced SQL and Big Data built-in functions.
- Strong programming skills in Python for data engineering tasks, including PySpark and DAG development.
- Extensive experience with cloud-based data platforms such as:
- AWS: S3, Glue, Lambda, Redshift, EMR,Athena
- Azure: Data Lake, Data Factory, Synapse
- GCP: BigQuery, Dataflow, Dataproc, Dataform, Cloud Storage
- Must have Agile project experience and hands-on experience with the JIRA tool.
- Hands-on experience in building data warehouses using DBT, BigQuery, and Databricks.
- Solid understanding of ETL/ELT processes and tools.
- Experience with version control systems (Github).
- Excellent problem-solving skills and the ability to troubleshoot complex data issues.
- Strong communication and interpersonal skills to collaborate effectively with cross-functional teams.

Key Skills

- Python | PySpark | SQL | BigQuery | Databricks | DBT | AWS | Azure | GCP | Data Modeling | ETL/ELT | Data Warehousing | Terraform | Jenkins | Tekton | GitHub Actions | Data Governance | CI/CD | Real-Time Data Processing | Cloud Data Engineering | Airflow | Astro

📌 Data Engineer (Chengalpattu)
🏢 Mahindra and Mahindra Limited [Automotive and Farm Equipment Business]
📍 Chengalpattu