Virtual Drive Next week:

Experience: 7- 10

Hyderabad

Notice period Immediate joiner

1

Role**

GCP Data Engineer Postgres Python Pyspark Developer (BigQuery, Cloud Storage, Dataproc, Airflow)

2

Required Technical Skill Set**

GCP Data Engineer to design, build, and optimize scalable data pipelines and analytics solutions using BigQuery, Cloud Storage, Dataproc, and Airflow.

3

No of Requirements**

5

4

Desired Experience Range**

7+ Years

5

Location of Requirement

HYDERABAD

6

Immediate Joiners Needed

YES

7

Work Timings

Desired Competencies (Technical/Behavioral Competency)

Must-Have**

(Ideally should not be more than 3-5)

- GCP Services: BigQuery, Cloud Storage, Dataproc, Cloud Composer (managed Airflow) or self-managed Airflow.
- Airflow: Solid experience in DAG creation, operators/hooks, scheduling, backfilling, retry strategies, and CI/CD for DAG deployments.
- Programming: Proficiency in Postgres Programming including Tables , Triggers, Views, Stored procedures, Python and Pyspark (PySpark, Airflow DAGs), SQL (advanced BigQuery SQL).
- Data Modeling: Dimensional modeling (Star/Snowflake), data vault basics, and schema design for analytics.




- Performance Tuning: BigQuery partitioning/clustering, predicate pushdown, job stats review, Dataproc executor tuning.
- Version Control & CI/CD: Git, branching strategies, pipelines for deploying Airflow DAGs and config.
- Operational Excellence: Monitoring with Stackdriver/Cloud Logging, debugging pipeline failures, and root-cause analysis.
- involves end-to-end ownership of data ingestion, transformation, orchestration, and performance tuning for batch and near real-time workflows.

Good-to-Have

- Streaming: Pub/Sub, Dataflow (Apache Beam) for near real-time pipelines.
- Orchestration Patterns: Event-driven pipelines, dependency management, and cross-environment promotion.
- Data Governance: Catalog/lineage tools (e.g., Data Catalog), PII handling, row-level security, column-level encryption.
- Containers & Infra: Docker, Terraform for IaC on GCP; Kubernetes concepts.
- BI Integration: Experience integrating with Looker, Tableau, or Power BI.
- Certifications: Google Professional Data Engineer / Cloud Architect.

📌 Postgresql Developer (Hyderabad)
🏢 Tata Consultancy Services
📍 Hyderabad