Overall Exp-5

Location-Pan India

Job Requirements*
Proficiency in programming languages: Python, Java
Expertise in data processing frameworks: Apache Beam (Data Flow)
Active experience on GCP tools and technologies like Big Query, Dataflow, Cloud Composer, Cloud Spanner, GCS, DBT etc.,
Data Engineering skillset using Python, SQL
Experience in ETL (Extract, Transform, Load) processes
Knowledge of DevOps tools like Jenkins, GitHub, Terraform is desirable. Should have valuable knowledge on Kafka (Batch/ streaming)
Understanding of Data models and experience in performing ETL design and build, database replication using Message based CDC
Familiarity with cloud storage solutions
Solid problem-solving abilities in data engineering challenges
Understanding of data security and scalability
- Proficiency in relevant tools like Apache Airflow

Core Skills:
GCP Services: BigQuery, Dataflow, Cloud Storage, Pub/Sub, Data Catalog, Data Fusion, AI Platform.
Programming: SQL, Python, Java/Scala (for Dataflow/Spark).
Data Concepts: Data Warehousing, ETL,



Data Modeling, Streaming.
Tools: Apache Beam, Docker, Kubernetes, Git, Terraform.

Key Responsibilities*
Design & Build Data Pipelines: Develop scalable data ingestion, transformation (ETL/ELT), and orchestration workflows using GCP services.
GCP Service Utilization: Implement solutions with BigQuery, Dataflow, Dataproc, Cloud Storage, Pub/Sub, and Cloud Composer.
Data Integration: Connect and manage data from diverse sources (APIs, databases, files).
Optimization: Improve performance, cost-efficiency, and reliability of data processes.
Collaboration: Work with data scientists, analysts, and architects to meet data needs.
Quality & Security: Ensure data integrity, security, and compliance.
Automation: Implement CI/CD and Infrastructure as Code (IaC) using tools like Terraform.
Monitoring & Support: Maintain, monitor, and troubleshoot data workloads and systems.

📌 Gcp Data Engineer Bengaluru (India)
🏢 Tata Consultancy Services
📍 India