JD:
Data Pipeline Development & Orchestration -Design, develop, and orchestrate data pipelines using Apache Airflow (Cloud Composer) to automate data ingestion, transformation, and loading (ETL/ELT) workflows.
Data Transformation with DBT -Implement DBT (Data Build Tool) models for transforming raw data into analytics-ready datasets in BigQuery, applying SQL-based transformations, modular modeling, and version control.
Stream & Batch Processing with Dataflow - Build real-time (Streaming) and batch data processing pipelines using Apache Beam on Dataflow, ensuring scalable and productive data processing.
BigQuery Optimization & Performance Tuning - Design optimized BigQuery schemas, implement partitioning, clustering, and tune queries for cost-efficiency and performance.
Monitoring, Logging, and CI/CD for Data Pipelines - Set up monitoring with Stackdriver, logging, error handling, and CI/CD pipelines for automated testing and deployment of DBT, Airflow DAGs, and Dataflow jobs.

📌 Gcp Engineer Bengaluru
🏢 Tekskills India Private
📍 Bengaluru