Information Details
1 Role GCP Data Engineer Postgres Python Pyspark Developer (Big Query, Cloud Storage, Dataproc, Airflow)
2 Required Technical Skill Set GCP Data Engineer to design, build, and optimize scalable data pipelines and analytics solutions using Big Query, Cloud Storage, Dataproc, and Airflow.
3 No of Requirements 5
4 Desired Experience Range 7+ Years
5 Location of Requirement HYDERABAD
6 Immediate Joiners Needed YES

Desired Competencies (Technical/Behavioral Competency)
Must-Have (Ideally should not be more than 3-5)
· GCP Services: Big Query, Cloud Storage, Dataproc, Cloud Composer (managed Airflow) or self-managed Airflow.
· Airflow: Strong experience in DAG creation, operators/hooks, scheduling, backfilling, retry strategies, and CI/CD for DAG deployments.
· Programming: Proficiency in Postgres Programming including Tables, Triggers, Views, Stored procedures, Python and Pyspark (PySpark, Airflow DAGs), SQL (advanced BigQuery SQL).
· Data Modeling: Dimensional modeling (Star/Snowflake), data vault basics, and schema design for analytics. ·
Performance Tuning:



BigQuery partitioning/clustering, predicate pushdown, job stats review, Dataproc executor tuning. ·

Version Control & CI/CD: Git, branching strategies, pipelines for deploying Airflow DAGs and config.
· Operational Excellence: Monitoring with Stackdriver/Cloud Logging, debugging pipeline failures, and root-cause analysis. · involves end-to-end ownership of data ingestion, transformation, orchestration, and performance tuning for batch and near real-time workflows.

Valuable-to-Have
· Streaming: Pub/Sub, Dataflow (Apache Beam) for near real-time pipelines.
· Orchestration Patterns: Event-driven pipelines, dependency management, and cross-environment promotion.
· Data Governance: Catalog/lineage tools (e.g., Data Catalog), PII handling, row-level security, column-level encryption.
· Containers & Infra: Docker, Terraform for IaC on GCP; Kubernetes concepts.
· BI Integration: Experience integrating wi

📌 GCP Data Engineer Postgres Python Pyspark Developer (Hyderabad)
🏢 Tata Consultancy Services
📍 Hyderabad