Key Responsibilities
• Design, develop, and maintain scalable data pipelines on GCP.
• Build batch and real-time data processing solutions using Dataflow and Pub/Sub.
• Develop and optimize data warehouse solutions using BigQuery.
• Build ETL/ELT workflows for structured and semi-structured data.
• Develop data processing scripts and applications using Python and SQL.
• Work with Cloud Storage for ingestion and data-lake use cases.
• Perform data transformation, validation, cleansing, and quality checks.
• Optimize BigQuery queries, partitioning, clustering, and overall pipeline performance.
• Implement workflow orchestration using tools such as Cloud Composer / Apache Airflow.
• Implement monitoring, logging, error handling, and production support processes.
• Collaborate with data architects, analysts, application teams, and DevOps teams.
• Follow security and access-control practices using IAM and service accounts.