Overview:
We are looking for an experienced Data Engineer to design, build, and maintain scalable, reliable, and high-performance data platforms. You will own data pipelines end to end, from ingestion and transformation to modeling, quality, and delivery, enabling analytics, business intelligence, and AI/ML initiatives across the organization.
The ideal candidate has robust hands-on expertise in Python and SQL, deep knowledge of modern data engineering practices, and the ability to independently architect and deliver data solutions on cloud platforms, with Google Cloud Platform (GCP) preferred.

Key Responsibilities:

Data Pipeline Development
• Design, build, and maintain scalable batch and real-time data pipelines using ETL/ELT patterns.
• Ingest and integrate structured, semi-structured, and unstructured data from diverse sources, including APIs, databases, files, and event streams.
• Implement incremental loading, change data capture (CDC), and idempotent pipeline design.
• Implement CDC pipelines using Datastream to replicate data from operational databases into BigQuery.




• Build batch and streaming pipelines with Apache Beam on Dataflow, using Pub/Sub for event ingestion.

Data Modeling and Architecture
• Design and optimize data models (dimensional modeling, star/snowflake schemas, Data Vault where relevant) for analytics and reporting.
• Build and manage data warehouses and lakehouses following the Medallion architecture (Bronze, Silver, Gold layers), ideally on BigQuery and Cloud Storage.
• Design BigQuery datasets using partitioning, clustering, materialized views, and authorized views for performance and secure sharing.
• Work with modern table formats such as Delta Lake, Apache Iceberg (including BigLake), or Apache Hudi.
• Contribute to data platform architecture, technology selection, and design decisions.

Orchestration and Transformation
• Orchestrate workflows using tools such as Cloud Composer (Apache Airflow), Dagster, Prefect, or other cloud-nati

📌 Data Engineer (Surat)
🏢 Blue Data Consulting
📍 Surat