Build batch and streaming data ingestion pipelines using Pub/Sub, Dataflow, Datastream (CDC), and Storage Transfer Service
Load and ingest data into BigQuery via batch loads, the BigQuery Storage Write API and external/BigLake tables
Manage Cloud Storage (GCS) landing zones, buckets, lifecycle policies and storage classes
Develop ETL/ELT workflows using Dataflow (Apache Beam), Dataproc (Spark/Hadoop) and Dataprep
Build SQL-based transformations using Dataform and BigQuery scheduled queries
Implement medallion/layered architecture (raw staging curated/gold)
Design and optimize BigQuery data warehouses with partitioning, clustering, materialized views and cost controls
Orchestrate pipelines using Cloud Composer (Airflow) and Workflows
Schedule and automate jobs with Cloud Scheduler and Cloud Functions
Manage access and security using IAM, service accounts and Workload Identity Federation
Implement data governance, lineage and quality using Knowledge Catalog
Apply encryption (CMEK/CSEK, Cloud KMS),



VPC Service Controls and DLP via Sensitive Data Protection
Ensure compliance with PII masking and column/row-level security in BigQuery
Enable analytics consumption through Looker, Looker Studio and BigQuery BI Engine
Monitor pipelines and resources with Cloud Monitoring, Cloud Logging, and Cloud Trace
Optimize query performance, slot usage, and overall cloud cost
Implement CI/CD using Cloud Build, Artifact Registry, and Cloud Source Repositories
Manage infrastructure-as-code using Terraform and Deployment Manager
Maintain version control, automated testing and pipeline reliability with alerting and SLAs

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Gcp Data Engineer Hyderabad
🏢 Paltech
📍 Hyderabad