Role & responsibilities

Must-Have •

Build, maintain, and troubleshoot pipelines using GCP services (e.g., DataProc, Pub/Sub, Cloud Functions, Cloud Composer/Apache Airflow) to ingest, transform, and load data from various sources (relational databases, APIs, streaming data, flat files).

• Implement batch data processing solutions.

• Identify and resolve data-related issues, including data quality problems, pipeline failures, and performance bottlenecks.

• Sound programming knowledge on PySpark & SQL in terms of processing large amount of semi structured & unstructured data • Working Knowledge on working with Avro, Parquet format files • Knowledge on working on Hadoop Big Data platform and ecosystem

Valuable-to-Have • Knowledge on Jira, Agile, Sonar, Team city & CICD • Any exposure / experience for an international Banking client / multi-vendor / multi geography teams





SN Responsibility of / Expectations from the Role

1 • Design, implement, and optimize scalable, reliable, and secure data architectures on GCP, including data lakes, data warehouses, and streaming solutions. • Develop and maintain data models for optimal storage, retrieval, and analysis. • Build, maintain, and troubleshoot robust ETL/ELT pipelines using GCP services (e.g., Dataflow, Pub/Sub, Cloud Functions, Cloud Composer/Apache Airflow) to ingest, transform, and load data from various sources (relational databases, APIs, streaming data, flat files). •

2 • Testing & Validation

3 • Debugging & troubleshooting

Type Details of The Role (For

📌 Gcp Data Engineer (Hyderabad)
🏢 Tata Consultancy Services
📍 Hyderabad