Build batch and streaming data ingestion pipelines using Pub/Sub, Dataflow, Datastream (CDC), and Storage Transfer Service

Load and ingest data into BigQuery via batch loads, the BigQuery Storage Write API and external/BigLake tables

Manage Cloud Storage (GCS) landing zones, buckets, lifecycle policies and storage classes

Develop ETL/ELT workflows using Dataflow (Apache Beam), Dataproc (Spark/Hadoop) and Dataprep

Build SQL-based transformations using Dataform and BigQuery scheduled queries

Implement medallion/layered architecture (raw → staging → curated/gold)

Design and optimize BigQuery data warehouses with partitioning, clustering, materialized views and cost controls

Orchestrate pipelines using Cloud Composer (Airflow) and Workflows

Schedule and automate jobs with Cloud Scheduler and Cloud Functions

Manage access and security using IAM, service accounts and Workload Identity Federation





Implement data governance, lineage and quality using Knowledge Catalog

Apply encryption (CMEK/CSEK, Cloud KMS), VPC Service Controls and DLP via Sensitive Data Protection

Ensure compliance with PII masking and column/row-level security in BigQuery

Enable analytics consumption through Looker, Looker Studio and BigQuery BI Engine

Monitor pipelines and resources with Cloud Monitoring, Cloud Logging, and Cloud Trace

Optimize query performance, slot usage, and overall cloud cost

Implement CI/CD using Cloud Build, Artifact Registry, and Cloud Source Repositories

Manage infrastructure-as-code using Terraform and Deployment Manager

Maintain version control, automated testing and pipeline reliability with alerting and SLAs

📌 GCP Data Engineer (Hyderabad)
🏢 Paltech
📍 Hyderabad