16 Sep
|
Tata Consultancy Services
|
Vasanthanagar
16 Sep
Tata Consultancy Services
Vasanthanagar
Overall Exp-5+
Location-Pan India
Job Requirements*
- Proficiency in programming languages: Python, Java
- Expertise in data processing frameworks: Apache Beam (Data Flow)
- Active experience on GCP tools and technologies like Big Query, Dataflow, Cloud Composer, Cloud Spanner, GCS, DBT etc.,
- Data Engineering skillset using Python, SQL
- Experience in ETL (Extract, Transform, Load) processes
- Knowledge of DevOps tools like Jenkins, GitHub, Terraform is desirable. Should have good knowledge on Kafka (Batch/ streaming)
- Understanding of Data models and experience in performing ETL design and build, database replication using Message based CDC
- Familiarity with cloud storage solutions
- Robust problem-solving abilities in data engineering challenges
- Understanding of data security and scalability
- Proficiency in relevant tools like Apache Airflow
Core Skills:
GCP Services : BigQuery, Dataflow, Cloud Storage , Pub/Sub , Data Catalog , Data Fusion , AI Platform .
Programming : SQL, Python, Java /Scala (for Dataflow/Spark).
Data Concepts : Data Warehousing,
ETL, Data Modeling, Streaming .
Tools : Apache Beam , Docker , Kubernetes , Git , Terraform.
Key Responsibilities*
Design & Build Data Pipelines : Develop scalable data ingestion, transformation (ETL/ELT), and orchestration workflows using GCP services.
GCP Service Utilization : Implement solutions with BigQuery, Dataflow , Dataproc , Cloud Storage , Pub/Sub , and Cloud Composer .
Data Integration : Connect and manage data from diverse sources (APIs, databases, files).
Optimization : Improve performance, cost-efficiency, and reliability of data processes.
Collaboration : Work with data scientists, analysts, and architects to meet data needs.
Quality & Security : Ensure data integrity, security, and compliance.
Automation : Implement CI/CD and Infrastructure as Code (IaC) using tools like Terraform.
Monitoring & Support : Maintain, monitor, and troubleshoot data workloads and systems.
📌 GCP Data Engineer (Vasanthanagar)
🏢 Tata Consultancy Services
📍 Vasanthanagar