Experience

410 Years

Role

Data Engineer – GCP / AWS

Job Summary

We are looking for an experienced Data Engineer with robust hands-on expertise in Python and SQL and experience working with GCP and/or AWS cloud platforms.

The ideal candidate should have strong experience in designing, developing, and optimizing data pipelines, ETL/ELT processes, data transformation, and cloud-based data solutions.

Mandatory Skills

- Strong hands-on experience in Python
- Strong proficiency in SQL
- Experience in Data Engineering / ETL / ELT
- Hands-on experience in GCP or AWS
- Strong understanding of data pipelines and data processing
- Experience working with large datasets
- Positive understanding of data modelling and database concepts

GCP Skills – Preferred

Experience with one or more of the following:

- Google BigQuery
- Google Cloud Storage (GCS)
- Dataflow
- Dataproc
- Pub/Sub
- Cloud Functions
- Composer / Airflow
- Cloud Run

AWS Skills – Preferred

Experience with one or more of the following:

- AWS S3
- AWS Glue
- EMR
- Lambda
- Redshift
- Athena
- Step Functions
- MWAA / Airflow

Data Engineering Responsibilities

- Design, develop,



and maintain scalable ETL/ELT data pipelines.
- Develop data transformation and processing solutions using Python and SQL.
- Build and optimize batch and/or real-time data processing pipelines.
- Develop data ingestion pipelines from multiple source systems.
- Perform data cleansing, transformation, validation, and aggregation.
- Work with cloud data platforms such as GCP and/or AWS.
- Optimize SQL queries and data processing jobs for performance.
- Work with data warehouses, data lakes, and cloud storage platforms.
- Monitor pipeline performance and troubleshoot production issues.
- Implement data quality and validation checks.
- Collaborate with business, application, and analytics teams to understand data requirements.
- Follow best practices for security, scalability, reliability, and maintainability.

Good to Have

- PySpark / Apache Spark
- Airflow / Cloud Composer
- Kafka / Pub/Sub
- BigQuery / Snowflake / Redshift
- Databricks
- Terraform
- CI/CD
- Docker / Kubernetes
- Git
- Data modelling
- Cloud migration experience

📌 Data Engineer (Chennai)
🏢 ALTIMETRIK
📍 Chennai