21 Aug
|
Skysoft
|
Gurugram
THE ROLE
We are building an enterprise data lake and analytics platform on Google Cloud for a large asset management firm. The platform does not exist yet — you would be one of the first engineers on it, setting up ingestion, modeling and orchestration from scratch and defining the standards the rest of the team builds to. The stack is BigQuery, Python, dbt and Cloud Composer (Airflow), with Terraform underneath. This is a hands-on engineering role with real ownership of architecture decisions.
REQUIRED SKILLS
Python — production-grade code, not scripts. Modules and packaging, testing (pytest), error handling and retries, API and file ingestion. pandas or polars for local processing.
SQL — advanced. Window functions, CTEs, complex joins, and the ability to reason about query cost and execution.
BigQuery — partitioning and clustering, incremental load patterns, materialized and authorized views, slots versus on-demand, and cost control. Equivalent depth in Snowflake, Redshift or Databricks SQL is acceptable.
dbt — layered project structure, generic and singular tests,
incremental models and strategies, snapshots for SCD Type 2, macros, and documentation.
Airflow / Cloud Composer — DAG design, dependencies, sensors, retries, idempotency, backfills and SLAs.
Data modeling — dimensional modeling, fact and dimension design, choosing and defending a grain, slowly changing dimensions, and handling late-arriving and restated data.
Google Cloud fundamentals — Cloud Storage and data lake layout, Parquet and Avro, IAM, service accounts, Secret Manager.
Engineering practice — Git, code review, CI/CD, and infrastructure as code with Terraform.
Data quality — designing checks, reconciliation against source systems, monitoring and alerting.
Communication — able to work directly with business users to define requirements and translate them into models.
WHAT YOU WILL DO
Build batch and CDC ingestion from portfolio accounting, order management, custodian, administrator and marke
📌 Senior Data Engineer (Gurugram)
🏢 Skysoft
📍 Gurugram