? We're Hiring: Senior Data Engineer

? Location: Bangalore

? Experience: 6+ Years

About the Role

We're looking for a Senior Data Engineer with strong data architecture and system design skills to design, build, and own end-to-end production pipelines powering analytics and decision-making across the organization. The role centers on deep Python and SQL expertise, robust transaction-level fact tables, and rigorous data quality and auditability, ideally within a finance, risk, or compliance data domain.

What You'll Do

? Own data architecture and system design decisions for pipelines and data models, including grain, partitioning, schema evolution, and scalability trade-offs.

? Take end-to-end ownership of production pipelines: design, build, deploy, monitor, and operate.

? Design and build transaction tables (append-only, immutable event/transaction-grain fact tables) alongside standard fact/dimension models using medallion (bronze/silver/gold) architecture.

? Build and support pipelines for finance, risk, or compliance use cases where accuracy, auditability, and data lineage are critical.

? Implement data quality and auditability controls, including validation checks, reconciliation logic, anomaly detection, and audit trails for every pipeline.

? Write complex, cross-dialect SQL (MySQL + PostgreSQL), including window functions, multi-layered CTEs, CASE-driven logic, COALESCE/NULLIF, type casting, and date/timestamp handling.

? Build idempotent reload patterns (DELETE + INSERT), UNION ALL/set operations, and templated (Jinja-style) SQL, including handling late-arriving/corrected transaction records without double-counting.

? Orchestrate Airflow-style DAGs with cross-pipeline dependencies and secure credential handling.

? Build Python ETL tooling using pandas (CSV ↔ DB), SQLAlchemy + raw drivers (MySQLdb, psycopg2),



type hints/dataclasses, and class-based pipeline design.

? Integrate with Google Drive/Sheets APIs, build YAML-driven pipeline configurations, and handle pipeline staging requirements.

? Own CI/CD for pipelines through standard Git workflows.

? Document data models and pipeline logic for maintainability, keeping compliance and audit requirements in mind.

What We're Looking For

Must-Have Skills:

✅ 6+ years of experience in data engineering.

✅ Strong data architecture and system design skills to make sound trade-offs on modeling, partitioning, and scalability—not just write queries.

✅ Demonstrated end-to-end production ownership of data pipelines from design through deployment, monitoring, and operational support.

✅ Expert-level Python and SQL as the primary toolset. Hive/Hadoop is a nice-to-have only.

✅ Robust data quality and auditability practices, including validation, reconciliation, anomaly detection, and audit trails.

✅ Experience working with finance, risk, or compliance data, with an understanding of accuracy, auditability, and data-lineage requirements.

✅ Proven experience designing and building transaction/fact tables.

✅ Expert SQL skills, including window functions, CTEs, CASE logic, and cross-dialect MySQL/PostgreSQL.

✅ Solid Python skills with pandas, SQLAlchemy, type hints/dataclasses, and class-based pipelines.

✅ Airflow-like orchestration experience.

✅ Solid data modeling fundamentals, including fact/dimension models, normalization, and medallion architecture.

✅ YAML-based configuration experience, along with standard CI/CD and Git workflows.

Nice-to-Have Skills

? Hive/Hadoop or Spark experience.

? Interested candidates: Share your updated resume

#Hiring #SeniorDataEngineer #DataEngineering #Python #SQL #DataArchitecture #ETL #DataPipelines #BangaloreJobs #TechHiring

📌 Senior Data Engineer (Bengaluru)
🏢 ShimentoX Technologies
📍 Bengaluru