Job Summary
JD & Role Summary
Design and deliver cloud-native data provisioning, subsetting, and refresh capabilities for GCP lower environments. Build secure and scalable pipelines using BigQuery and associated GCP services to extract, transform, protect, load, validate, and refresh test datasets. Optimize performance and cost while ensuring that data movement, storage, lineage, and access comply with privacy, security, and data-governance requirements.
Key Responsibilities
- Design and build automated provisioning pipelines for GCP and BigQuery lower environments.
- Implement metadata-driven extraction, subsetting, transformation, loading, and refresh workflows.
- Integrate TDM masking and synthetic-data services into end-to-end data pipelines.
- Develop incremental and full refresh patterns with scheduling, restartability, reconciliation, and exception handling.
- Optimize BigQuery SQL, partitioning, clustering, storage, and compute utilization.
- Implement secure service integration, access controls, encryption, logging, lineage, and audit evidence.
- Validate record counts, schema consistency, referential integrity, and data quality after provisioning.
- Support workplace onboarding, performance testing, release deployment, and operational knowledge transfer.
Skills
- GCP
- BigQuery
- SQL and Performance Tuning
- Python
- Data Pipelines and Orchestration
- Data Subsetting
- Automated Refresh
- Cloud Security and IAM
Experience Level
3-10 Years
Mode Of Hiring
Key Responsibilities
JD & Role Summary
Design and deliver cloud-native data provisioning, subsetting, and refresh capabilities for GCP lower environments.
Build secure and scalable pipelines using BigQuery and associated GCP services to extract, transform, protect, load, validate, and refresh test datasets. Optimize performance and cost while ensuring that data movement, storage, lineage, and access comply with privacy, security, and data-governance requirements.
Key Responsibilities
- Design and build automated provisioning pipelines for GCP and BigQuery lower environments.
- Implement metadata-driven extraction, subsetting, transformation, loading, and refresh workflows.
- Integrate TDM masking and synthetic-data services into end-to-end data pipelines.
- Develop incremental and full refresh patterns with scheduling, restartability, reconciliation, and exception handling.
- Optimize BigQuery SQL, partitioning, clustering, storage, and compute utilization.
- Implement secure service integration, access controls, encryption, logging, lineage, and audit evidence.
- Validate record counts, schema consistency, referential integrity, and data quality after provisioning.
- Support environment onboarding, performance testing, release deployment, and operational knowledge transfer.
Skills
- GCP
- BigQuery
- SQL and Performance Tuning
- Python
- Data Pipelines and Orchestration
- Data Subsetting
- Automated Refresh
- Cloud Security and IAM
Experience Level
3-10 Years
Mode Of Hiring
Skill Requirements
JD & Role Summary
Design and deliver cloud-native data provisioning, subsetting, and refresh capabilities for GCP lower environments. Build secure and scalable pipelines using BigQuery and associated GCP services to extract, transform, protect, load, validate, and refresh test datasets. Optimize performance and cost while ensuring that data movement, storage, lineage, and access comply with privacy, security, and data-governance requirements.
Key Responsibilities
- Design and build automated provisioning pipelines for GCP and BigQuery lower environments.
- Implement metadata-driven extraction, subsetting, transformation, loading, and refresh workflows.
- Integrate TDM masking and synthetic-data services into end-to-end data pipelines.
- Develop incremental and full refresh patterns with scheduling, restartability, reconciliation, and exception handling.
- Optimize BigQuery SQL, partitioning, clustering, storage, and compute utilization.
- Implement secure service integration, access controls, encryption, logging, lineage, and audit evidence.
- Validate record counts, schema consistency, referential integrity, and data quality after provisioning.
- Support workplace onboarding, performance testing, release deployment, and operational knowledge transfer.
Skills
- GCP
- BigQuery
- SQL and Performance Tuning
- Python
- Data Pipelines and Orchestration
- Data Subsetting
- Automated Refresh
- Cloud Security and IAM
Experience Level
3-10 Years
Mode Of Hiring
Must Have Skill
Data Pipelines,GCP BigQuery,MySQL,Python
Good to Have Skill
Other Requirements
📌 Technical Specialist (India)
🏢 HCLTech
📍 India