29 Aug
|
Naukri e-Hire
|
Bengaluru
29 Aug
Naukri e-Hire
Bengaluru
We have an urgent job opening for our client Nineleaps Technology Solutions Private Limited.
Exp: Min 5 yrs
Location: Bangalore (Sarjapur Road)
Key skills:Data Engineer, Python, SQL, Systen Design Architecture, Financial Transaction Data etc.
About the Role: We're looking for a Senior Data Engineer with strong data architecture and system design skills to design, build, and own end-to-end production pipelines powering analytics and decision-making across the org. The role centers on deep Python + SQL expertise, robust transaction-level fact tables, and rigorous data quality and auditability, ideally within a finance, risk, or compliance data domain. Internal candidates familiar with Uber's data platform stack (Piper/uWorc, Databook, uSecret, DSW, SourceGraph, Query Builder, OneETL) will be prioritized.
Location: Bangalore
Experience: 6+ Years
What You'll Do
Own data architecture and system design decisions for pipelines and data models - grain, partitioning, schema evolution, and scalability tradeoffs.
Take end-to-end ownership of pipelines in production: design, build, deploy, monitor, and operate.
Design and build transaction tables (append-only, immutable event/transaction-grain fact tables) alongside standard fact/dimension models using medallion (bronze/silver/gold) architecture.
Build and support pipelines for finance, risk, or compliance use cases where accuracy, auditability, and data lineage are critical.
Implement data quality and auditability controls: validation checks, reconciliation logic, anomaly detection, and audit trails for every pipeline you own.
Write complex, cross-dialect SQL (MySQL + PostgreSQL): window functions, multi-layered CTEs, CASE-driven logic, COALESCE/NULLIF, type casting, and date/timestamp handling.
Build idempotent reload patterns (DELETE+INSERT), UNION ALL/set operations, and templated (Jinja-style) SQL including handling for late-arriving/corrected transaction records without double-counting.
Orchestrate Airflow-style DAGs (Pipeline/BaseTask,
ExternalTaskSensor for cross-pipeline deps) with secure credential handling via uSecret.
Build Python ETL tooling: pandas (CSV DB), SQLAlchemy + raw drivers (MySQLdb, psycopg2), type hints/dataclasses, class-based pipeline design.
Integrate with Google Drive/Sheets APIs; build YAML-driven pipeline configs; handle Piper staging/user_staging quirks.
Own CI/CD for pipelines via standard Git workflows.
Use Uber's internal stack day-to-day: OneETL for pipeline authoring, Piper/uWorc for scheduling, Databook for lineage, uSecret for credentials, DSW notebooks for CSV/table work, SourceGraph for code search, Query Builder for ad hoc SQL. Document data models and pipeline logic for maintainability, keeping compliance/audit needs in mind.
What We're Looking For
Must-Have:
6+ years in data engineering.
Robust data architecture and system design skills to make sound tradeoffs on modeling, partitioning, and scalability, not just write queries.
Demonstrated end-to-end production ownership of data pipelines from design through deployment, monitoring, and operational support.
Expert-level Python + SQL as the primary toolset.
Hive/Hadoop is a nice-to-have only.
Robust data quality and auditability practices - validation, reconciliation, anomaly detection, audit trails.
Experience working with finance, risk, or compliance data - comfort with the heightened accuracy, auditability, and data-lineage requirements typical of these domains.
Proven experience designing and building transaction/fact tables.
Expert SQL (window functions, CTEs, CASE logic, cross-dialect MySQL/PostgreSQL).
Strong Python (pandas, SQLAlchemy, type hints/dataclasses, class-based pipelines).
Airflow-like orchestration experience (Piper/uWorc a plus).
Solid data modeling fundamentals (fact/dim, normalization, medallion architecture).
YAML-based config experience; standard CI/CD and Git workflows. Strongly Preferred (Uber Internal Tooling): Piper/uWorc, Databook, uSecret, DSW, SourceGraph, Query Builder, OneETL.
Nice-to-Have: Hive/Hadoop or Spark experience.
📌 Sr. Data Engineer (Bengaluru)
🏢 Naukri e-Hire
📍 Bengaluru