Role & responsibilities

GCP Engineering Skills

- Google Cloud Platform (GCP)

- BigQuery

- Dataproc / Apache Spark

- Bigtable

- Dataplex / Data Catalog
- Python (ETL/ELT pipelines and automation)
- SQL (data transformation, validation, reconciliation)

- Databases

- MySQL

- Hive

- MongoDB
- Cloud Composer / Apache Airflow

- Kafka and Google Pub/Sub
- Apigee / REST APIs (data extraction and integration)
- GenAI (LLMs, embeddings, prompt engineering) and integration with data platforms
- Experience with Vertex AI, BigQuery ML, or GenAI-enabled GCP services (preferred)

Key Responsibilities Migration & Modernization

- Design and execute comprehensive migration strategies from on-premises, legacy platforms, or other cloud environments to GCP.
- Migrate structured and semi-structured data to BigQuery, Bigtable, and other GCP storage solutions.
- Build, optimize, and maintain scalable migration pipelines using Python, SQL, and Dataproc (Apache Spark).
- Implement batch and streaming migration patterns using:





- Cloud Composer (Apache Airflow)

- Kafka

- Google Pub/Sub

Data Quality & Governance

- Perform schema mapping, data transformation, reconciliation, and validation to ensure accuracy, consistency, and completeness of migrated datasets.
- Enable metadata management, lineage, and governance using Dataplex and Data Catalog.
- Handle large-volume data transfers using best practices including:

- Parallel loading

- Incremental loads

- CDC (Change Data Capture)

GenAI Adoption

- Leverage GenAI and automation techniques to accelerate:

- Data discovery

- Schema understanding

- Mapping recommendations

- Migration assessments
- Apply GenAI-assisted:

- Data transformation

- Code generation

- Migration pattern optimization

- SQL conversion

- Pipeline scaffolding

- Test case generation

Preferred candidate profile

📌 Python Software Developer (Hyderabad)
🏢 KRG Technologies
📍 Hyderabad