11 Sep
|
MindBrain
|
Chennai
GCP Test Engineer
Experience: 8+ Years
Location: Remote
Employment Type: Contract
Shift: 2:00 PM – 11:00 PM IST
Laptop: Candidate must be able to collect the company laptop from Mumbai / Bangalore / Chennai office locations.
Job Overview
We are looking for an experienced GCP Test Engineer with 8+ years of experience in data testing, ETL validation, cloud data warehouse testing, and data quality assurance. The ideal candidate should have strong hands-on experience validating GCP-based data pipelines using BigQuery, Dataflow, Cloud Composer (Airflow), and Google Cloud Storage (GCS).
Key Responsibilities
- Perform end-to-end validation of GCP-based data pipelines involving BigQuery, Dataflow, Cloud Composer, and GCS.
- Analyze complex BigQuery SQL queries for source-to-target reconciliation, data validation, and transformation testing.
- Validate historical, incremental, batch, and streaming data loads across cloud data warehouse environments.
- Perform comprehensive Data Quality Assurance (DQA) checks covering:
- Completeness
- Accuracy
- Consistency
- Uniqueness
- Freshness
- Referential Integrity
- Validate Apache Airflow DAGs in Cloud Composer, including dependencies, scheduling, workflow monitoring, and failure analysis.
- Perform data validation for Dataflow pipelines and investigate processing anomalies and data discrepancies.
- Validate fact and dimensional table loads in BigQuery.
- Test surrogate key generation and Slowly Changing Dimensions (SCD) Type 1 and Type 2 implementations.
- Develop and execute SQL-based data validation and audit reconciliation frameworks.
- Identify, analyze, and troubleshoot data quality issues and perform root cause analysis (RCA).
- Collaborate with Data Engineering and QA teams to ensure production data integrity.
- Participate in Agile ceremonies, defect triage, release validation, and production deployment verification.
- Prepare and maintain test cases, validation scripts, defect reports, and test documentation.
Mandatory Skills Candidates must have strong hands-on experience in:
1. Google BigQuery
2. Google Cloud Dataflow
3. Cloud Composer / Apache Airflow
4. Google Cloud Storage (GCS)
Required Technical Expertise
- 8+ years of experience in Data Testing / ETL Testing / Data Engineering QA
- Solid SQL skills, particularly BigQuery SQL
- Experience with source-to-target reconciliation
- Data warehouse and ETL/ELT testing
- Batch and streaming data validation
- Data quality and data integrity testing
- Dimensional and fact table testing
- SCD Type 1 & Type 2 validation
- Airflow DAG and workflow validation
- Cloud-based data pipeline testing
- Defect management and RCA
- Experience working in Agile/Scrum environments
Preferred Profile
- Strong understanding of GCP data architecture and data pipelines
- Experience working closely with Data Engineers and Cloud teams
- Knowledge of automated data validation frameworks
- Good analytical and troubleshooting skills
- Strong communication and coordination skills
Candidate Requirements
- Experience: 8+ Years
- Budget: ₹1.70 LPM
- Location: Remote
- Shift: 2:00 PM – 11:00 PM IST
- Laptop Collection: Mumbai / Bangalore / Chennai office
- Notice Period: Immediate / Short Notice preferred
- Contract Position
Must-Have Keywords for Sourcing BigQuery | Dataflow | Cloud Composer | Apache Airflow | GCS | SQL | ETL Testing | Data Testing | Data Quality | Data Validation | Source-to-Target Reconciliation | SCD Type 1 | SCD Type 2 | Data Warehouse Testing
📌 GCP Test Engineer (8+ yrs) (Chennai)
🏢 MindBrain
📍 Chennai