Data Engineer

Experience: 4–8 Years

Role: Data Engineer

Role Overview

We are looking for an experienced Data Engineer with 4–8 years of experience in designing, developing, and maintaining scalable data pipelines and data platforms. The ideal candidate should have robust hands-on experience with SQL, Python, PySpark, cloud-based data engineering, ETL tools, data warehousing, and orchestration frameworks.

The candidate will be responsible for building reliable data pipelines, integrating data from multiple sources, implementing data transformation processes, and supporting scalable data solutions across cloud environments.

Key Responsibilities

- Design, develop, and maintain scalable ETL/ELT data pipelines for processing structured and unstructured data.
- Develop data transformation and processing solutions using Python, SQL, and PySpark.
- Build and manage data pipelines using platforms such as Databricks, AWS Glue, and Snowflake.
- Work with cloud platforms including AWS, Azure, and/or GCP to develop and deploy data engineering solutions.
- Design and implement data warehouses, data marts, and data models to support analytical and reporting requirements.
- Work with databases and cloud data platforms such as BigQuery, Amazon Redshift, and SQL Server.
- Develop and manage data workflows and pipeline orchestration using Apache Airflow.
- Work with streaming and real-time data processing using Apache Kafka.
- Implement data quality, validation, monitoring, error handling, and performance optimization across data pipelines.




- Collaborate with BI and analytics teams to support reporting and visualization requirements using Power BI, Tableau, and Looker.
- Implement and maintain CI/CD practices for data pipelines, code, and deployments.
- Troubleshoot pipeline failures, data issues, and performance bottlenecks and provide effective solutions.
- Follow best practices related to data security, governance, scalability, reliability, and maintainability.
- Collaborate with cross-functional teams including Data Analysts, BI Developers, Software Engineers, and Business stakeholders.

Required Skills Programming & Data Processing

- Strong hands-on experience with SQL, Python, and PySpark.
- Good understanding of data processing, transformation, optimization, and automation.

ETL / Data Engineering Tools
- Experience with Databricks, AWS Glue, and Snowflake.
- Strong understanding of ETL/ELT concepts and pipeline development.

Cloud Platforms
- Hands-on experience with one or more major cloud platforms:

- AWS

- Microsoft Azure

- Google Cloud Platform (GCP)

Databases & Data Platforms
- Experience working with one or more of:

- BigQuery

- Amazon Redshift

- SQL Server
- Strong understanding of relational databases, data storage, querying, and optimization.

Data Warehousing & Data Modeling
- Strong understanding of Data Warehousing concepts.




- Experience with dimensional modeling, fact and dimension tables, star/snowflake schemas, and data marts.
- Understanding of data modeling and designing efficient analytical data structures.

Orchestration
- Hands-on experience with Apache Airflow.
- Understanding of DAG creation, scheduling, dependencies, monitoring, and error handling.

Messaging / Streaming
- Experience with Apache Kafka and understanding of event-driven or real-time data processing.

Reporting & Visualization
- Exposure to BI and reporting tools such as:

- Power BI

- Tableau

- Looker
- Ability to work with analytics/reporting teams to provide reliable and optimized datasets.

CI/CD
- Good understanding of CI/CD concepts and experience integrating data engineering workflows with deployment pipelines.
- Familiarity with version control, automated testing, and deployment practices.

Preferred Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
- Experience working on large-scale data engineering or cloud data platform projects.
- Strong problem-solving and analytical skills.
- Good understanding of data quality, governance, security, and performance optimization.
- Strong communication and collaboration skills.

Key Technologies SQL | Python | PySpark | Databricks | AWS Glue | Snowflake | AWS | Azure | GCP | BigQuery | Redshift | SQL Server | Power BI | Tableau | Looker | Apache Airflow | Apache Kafka | CI/CD | Data Warehousing | Data Modeling

📌 Data Engineer (Chennai)
🏢 fluid.live
📍 Chennai