19 Nov
|
CloudBerry360
|
Bengaluru
19 Nov
CloudBerry360
Bengaluru
We are hiring a hands-on engineer to build data platform from scratch: an AWS lakehouse serving board
KPIs and embedded dashboards, in both the UK and Ireland.
Title - Lead Data Platform Engineer (hands-on)
Company - Cloudberry360
Location - Bangalore
Start - Immediate
About us and the work
Cloudberry360 is a digital product development and services company with teams in the UK and India.
We build product stacks for clients across several domains. You will join a large Cloudberry team already embedded with a fast-growing client operating in the UK and Ireland. The client runs a microservice platform (CRM, agent portal, identity) used by estate and letting agencies. You will be the first data platform hire and will build its data platform from the ground up. It will power board and branch KPIs internally, and embedded dashboards for each agency customer.
What you'll do
You will design the platform and write most of its first version yourself.
- Stand up the AWS foundations for the UK and Ireland in Terraform: accounts, S3, Glue Iceberg catalog, Lake Formation permissions
- Build change-data-capture pipelines from the microservice databases with AWS DMS, landing straight into the lakehouse
- Load external sources: PostHog product analytics and public property datasets (Price Paid, EPC,
BER, PPR)
- Model bronze, silver and gold layers in dbt, including incremental marts, history (SCD2) and tests
- Mask and minimise personal data, and enforce row and column level access for internal and agency users
- Deliver the first internal KPI dashboards and the first embedded, tenant-isolated agency dashboard
- Set up orchestration, data quality alerting, CI/CD and Iceberg table maintenance
- Agree KPI definitions with the client's product and business teams, and push back where needed
- Set engineering standards, then help hire and mentor the data team as it grows
Tech environment .
The target stack is set, but you will have a real say in the final choices.
Layer - Technology Ingestion AWS DMS (full load + CDC), scheduled batch loaders, PostHog batch exports
Storage - Amazon S3 + Apache Iceberg, AWS Glue catalog
Governance - AWS Lake Formation, IAM, two AWS regions (London, Ireland)
Transformation - dbt Core on Amazon Athena
Orchestration - Dagster or Amazon MWAA (Airflow)
BI - Amazon Quick Sight with embedded dashboards and tag-based row-level security
Engineering - Terraform, GitHub, CI/CD, Python, SQL
What you'll bring
Around 6–9 years in data engineering, with at least 2–3 years building up-to-date lakehouse or dbt platforms hands-on.
- Production AWS data work: S3, DMS, Glue, Lake Formation and IAM, ideally across more than one region or account
- Apache Iceberg (or Delta/Hudi) in production: table design, partitioning, compaction, snapshot expiry
- Deep dbt experience: incremental models, snapshots, tests and keeping a growing project tidy
- Excellent SQL and solid Python
- Terraform (or similar) and CI/CD for data projects
- An orchestrator: Dagster, Airflow or Prefect
- Change-data-capture pipelines from operational databases (Postgres or MySQL)
- Practical GDPR knowledge: personal data masking, data residency and least-privilege access
- Confidence with stakeholders: turning business questions into clear KPI definitions
- A track record of building something from zero, not only maintaining it
📌 Lead Data Platform Engineer (Bengaluru)
🏢 CloudBerry360
📍 Bengaluru