Senior Data Engineer –

Job Description
Location
- Bangalore Role Overview We are looking for a Senior Data Engineer who provides technical leadership and hands-on ownership for complex, high-impact data engineering solutions across client engagements. This role sits between Data Engineer and Principal Engineer, extending senior execution into cross-project design decisions, standards, and accelerators, while still delivering critical components end-to-end. The Principal Data Engineer reduces ambiguity, drives technical direction, and raises engineering quality across teams.

Key Responsibilities
Technical Leadership & Solution Design
• Lead design for complex data subsystems (ingestion, transformation, curation, serving).
•
• Drive key design choices such as batch vs streaming, lakehouse vs warehouse, orchestration approach, and partitioning strategy.
•
• Define and enforce standards for pipeline design, coding practices, documentation, and production readiness. Hands-on Data Engineering Ownership
• Design, build, and optimize ETL / ELT pipelines using Python, SQL, and distributed processing frameworks (Spark / Databricks).
•
• Own complex integrations across multiple source systems (APIs, files, CDC/event streams).
•
• Optimize pipelines for performance, reliability, and cost efficiency. Orchestration, Reliability & Operations
• Architect and implement orchestration patterns (Airflow or equivalent), including retries, backfills, SLAs, and dependency management.
•
• Establish monitoring, alerting, and data quality checks.
•
• Lead triage and resolution of high-severity data pipeline issues. Platform Enablement (Applied, Not Architectural-Heavy)
• Implement CI/CD practices for data pipelines.
•
• Build reusable components and templates (ingestion patterns,



validation utilities, standard DAGs) to accelerate delivery.
•
• Partner with Architects to align solutions with modern cloud data platform patterns where needed. Mentoring & Collaboration
• Mentor Senior and mid-level engineers through design and code reviews.
•
• Clearly communicate design decisions and trade-offs to engineers, product teams, and clients/stakeholder. Design & Architecture Articulation
• Produce and maintain design docs and Architecture Decision Records (ADRs) including trade-offs, risks, and rollout plans.
•
• Communicate design intent and responsibilities across teams clearly. Must-Have Skills
• Robust proficiency in Python and SQL.
•
• Deep hands-on experience with Spark / PySpark / Databricks.
•
• Solid experience designing production-grade ETL / ELT pipelines and orchestration workflows.
•
• Strong understanding of data modeling, data warehouses, and data lakes.
•
• Practical experience with cloud platforms (AWS / Azure / GCP).
•
• Proven ability to lead technical decisions, mentor engineers, and deliver under ambiguity. Nice-to-Have Skills
• Experience with streaming or event-driven data pipelines (Kafka, Kinesis, Pub/Sub).
•
• Exposure to data observability, governance, or catalog tools.
•
• Familiarity with containers, IaC, or platform enablement practices.
•
• Relevant cloud or data engineering certifications. What Success Looks Like in This Role
• You deliver the hardest and most ambiguous parts of client data platforms.
•
• You reduce ambiguity by turning requirements into clear designs and execution plans.
•
• Your pipelines are reliable, observable, cost-aware, and production-ready.
•
• You uplift the DE practice through standards, reusable components, and mentorship.

📌 Senior Data Engineer (India)
🏢 Zemoso Technologies
📍 India