- Build batch and streaming data ingestion pipelines using Pub/Sub, Dataflow, Datastream (CDC), and Storage Transfer Service
- Load and ingest data into BigQuery via batch loads, the BigQuery Storage Write API and external/BigLake tables
- Manage Cloud Storage (GCS) landing zones, buckets, lifecycle policies and storage classes
- Develop ETL/ELT workflows using Dataflow (Apache Beam), Dataproc (Spark/Hadoop) and Dataprep
- Build SQL-based transformations using Dataform and BigQuery scheduled queries
- Implement medallion/layered architecture (raw staging curated/gold)
- Design and optimize BigQuery data warehouses with partitioning, clustering, materialized views and cost controls
- Orchestrate pipelines using Cloud Composer (Airflow) and Workflows
- Schedule and automate jobs with Cloud Scheduler and Cloud Functions
- Manage access and security using IAM, service accounts and Workload Identity Federation
- Implement data governance, lineage and quality using Knowledge Catalog
- Apply encryption (CMEK/CSEK, Cloud KMS),



VPC Service Controls and DLP via Sensitive Data Protection
- Ensure compliance with PII masking and column/row-level security in BigQuery
- Enable analytics consumption through Looker, Looker Studio and BigQuery BI Engine
- Monitor pipelines and resources with Cloud Monitoring, Cloud Logging, and Cloud Trace
- Optimize query performance, slot usage, and overall cloud cost
- Implement CI/CD using Cloud Build, Artifact Registry, and Cloud Source Repositories
- Manage infrastructure-as-code using Terraform and Deployment Manager
- Maintain version control, automated testing and pipeline reliability with alerting and SLAs

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 GCP Data Engineer (Hyderabad)
🏢 Paltech
📍 Hyderabad