30 Oct
|
ValueLabs
|
Hyderabad
30 Oct
ValueLabs
Hyderabad
Dear Aspirants, Greetings from ValueLabs We are hiring for GCP Data engineer Role: GCP Data engineer
Experience:
6+ Years NP: Immediate
Location:
Hyderabad
- Hybrid
Job Description
Data Engineers build, test and run the data pipelines and models on our GCP platform. You will work with the Data Architect to turn designs into reliable, cost-efficient production pipelines that feed analytics, reporting and ML.
Key responsibilities
• Build batch and streaming pipelines using Dataflow (Apache Beam), Dataproc (PySpark), Pub/Sub and Datastream.
• Develop and optimise BigQuery datasets, tables, views and stored procedures; apply partitioning and clustering.
• Ingest data from databases, APIs, files and SaaS sources into Cloud Storage and BigQuery.
• Orchestrate and schedule workflows with Cloud Composer (Apache Airflow).
• Implement transformations and data models with SQL, Python, dbt or Dataform.
• Build data-quality checks, validation and reconciliation; monitor pipelines with Cloud Monitoring and Logging.
• Support migration of legacy ETL (Informatica, SSIS, Talend, Hadoop) workloads to GCP.
• Write infrastructure as code with Terraform and deploy through CI/CD pipelines.
• Apply security and governance standards: IAM roles, PII masking, Dataplex / Data Catalog metadata.
• Troubleshoot production issues, tune performance and reduce cost.
• Write technical documentation and take part in code reviews and Agile ceremonies. Required skills
• Hands-on experience with BigQuery, Dataflow or Dataproc, Pub/Sub, Cloud Storage and Cloud Composer.
• Solid SQL (complex joins, window functions, performance tuning).
• Strong Python; PySpark experience.
• Understanding of data warehousing, dimensional modelling and ETL / ELT patterns.
• Experience with Git and CI/CD (Cloud Build, GitHub Actions or Jenkins).
• Familiarity with Linux shell scripting and REST APIs.
• Good problem-solving and communication skills; comfortable working in Agile teams. Nice to have
• Java or Scala for Apache Beam.
• Experience with Kafka, Cloud Functions / Cloud Run, or Bigtable / Spanner / Cloud SQL.
• Exposure to Looker / Looker Studio, BigQuery ML or Vertex AI.
• Experience on AWS or Azure data services.
📌 GCP Data Engineer (Hyderabad)
🏢 ValueLabs
📍 Hyderabad