02 Oct
|
Coretek Services
|
Kondapur
02 Oct
Coretek Services
Kondapur
Description
Coretek is looking for a Data Engineer to build and operate the pipelines and data models that the rest of the business runs on. You'll own ingestion from source systems through to curated, well-documented datasets that analysts, data scientists, and application teams depend on. This is a hands-on engineering role: you'll write production code, design schemas, and be accountable for the reliability and cost of what you ship.
Responsibilities
- Design, build, and maintain batch and streaming data pipelines that are idempotent, observable, and recoverable.
- Model data for analytics (dimensional models, semantic layers, and curated marts), balancing query performance against maintainability.
- Integrate data from operational databases, SaaS APIs, files, and event streams, including handling schema drift and late-arriving data.
- Build data quality checks (freshness, volume, uniqueness, referential integrity) into pipelines rather than bolting them on afterward, and define how failures alert and escalate.
- Own pipelines in production: monitoring,
on-call rotation for data incidents, root-cause analysis, and backfills.
- Tune performance and cost (partitioning, clustering, file sizing, warehouse and cluster sizing) and make the tradeoffs explicit.
- Apply engineering discipline to data: version control, code review, CI/CD, automated testing, and infrastructure as code.
- Implement access controls, PII handling, retention, and lineage and audit requirements in partnership with security and compliance.
- Partner with analysts, data scientists, and product engineers to turn ambiguous requirements into durable data contracts.
- Maintain data dictionaries, lineage, and pipeline runbooks so consumers can find a dataset, understand what each field means and how current it is, and use it correctly without having to ask the team that built it.
Requirements
- 5+ years building production data pipelines.
- Robust hands-on Python development for data engineering, with
📌 Data Engineer (Kondapur)
🏢 Coretek Services
📍 Kondapur