Experience: 12 15+ years

Location: Pune (onsite)

About the Role

We are looking for a Principal Architect to lead the design and development of a Lakehouse solution end to end. This is an opportunity to architect and build a reusable data platform framework from the ground up.

You will own the technical architecture, write code for the core framework, mentor a growing engineering team, and work closely with leadership on the overall technology direction. The ideal candidate is a hands-on architect with a proven track record of building complex frameworks and platforms using open-source big-data technologies such as Spark, Hadoop, and modern Lakehouse stacks (e.g., Iceberg, Trino, Databricks, etc).

Key Responsibilities

Architecture (40%)
- Own the end-to-end architecture of the Lakehouse platform and its reusable framework components
- Design a metadata-driven data engineering framework built on Spark SQL and PySpark
- Architect a pipeline engine supporting dynamic SQL generation, incremental loads, CDC, SCD implementations, and schema evolution




- Design Kubernetes-native deployment with multi-tenancy, resource isolation, scaling strategy, high availability, and disaster recovery
- Define API contracts, plugin/extension architecture, and integration patterns
- Drive and document key architecture decisions (ADRs)

Hands-on Engineering (40%)
- Build core framework components: pipeline engine, energetic SQL generation, data quality checks, error handling and retry mechanisms
- Implement observability across the platform
- Optimize Spark job performance (partitioning, shuffle tuning, memory management, query plans)
- Establish and implement CI/CD pipelines, automated testing (unit and integration), and release practices
- Embed data governance and security into the platform (access control, auditability, cataloging)

Engineering Leadership (20%)
- Mentor and guide junior engineers; conduct design and code reviews
- Define coding standards, engineering best practices,

📌 Technical Architect (Pune)
🏢 Smart Analytica
📍 Pune