14 Nov
|
Sky Systems
|
India
About the Role
We are seeking a highly experienced Databricks Architect to lead the design and evolution of a up-to-date, enterprise-grade data platform built on the Databricks Lakehouse architecture . This is a senior-level role with a 50% hands-on coding and 50% architecture, design, and mentorship focus.
Key ResponsibilitiesArchitecture & Strategy
- Define the overall Databricks platform architecture, including workspace design, storage patterns, compute strategies, and enterprise integrations
- Establish architectural standards to ensure scalability, performance, security, and cost efficiency
Governance & Security
- Design and implement Unity Catalog governance frameworks , including:
- Metastore design
- Multi-catalog and schema strategies
- Fine-grained access controls and ABAC
- Identity federation and compliance requirements
Technical Leadership
- Act as the technical authority for the Databricks platform
- Perform code reviews and mentor senior and mid-level engineers
- Define best practices for:
- Medallion architecture
- Delta Lake optimization
- Data mesh and cross-domain data sharing
- Pipeline orchestration
Platform Design & Optimization
- Architect compute strategies (job clusters, all-purpose clusters, SQL warehouses, serverless)
- Drive performance tuning, cost optimization, and workload isolation
- Design monitoring and observability approaches
DevOps & Automation
- Design CI/CD pipelines for Databricks assets
- Implement Git-based development workflows
- Define infrastructure-as-code standards using Terraform or similar tools
Integration & Enablement
- Design integration patterns with upstream source systems and downstream analytics tools
- Translate business requirements into clear architectural blueprints
- Enable and upskill engineering teams through guidance and documentation
Required Skills & Experience
- 7+ years of experience in data architecture, data engineering, or platform engineering
- 3+ years of hands-on Databricks architecture experience
- Expert knowledge of:
- Databricks Lakehouse architecture
- Unity Catalog
- Delta Lake and Delta Live Tables
- Databricks Workflows and SQL Warehouses
- Databricks SQL and MLflow
- Strong proficiency in SQL, Python, and PySpark :
- Experience with enterprise data architectures (medallion, multi-hop, data mesh)
- Cloud platform experience (Azure, AWS, or GCP)
- Hands-on experience with Terraform or similar IaC tools :
- Proven background in performance tuning and cost optimization
Preferred Qualifications
- Databricks certifications (Data Engineer Qualified, Solutions Architect)
- Cloud certifications (Azure, AWS, or GCP)
- Experience with:
- Streaming platforms (Kafka, Event Hubs, Kinesis)
- MLOps and AI/ML platform architecture
- Multi-tenant and secure data architectures
- Analytics tools (Power BI, Tableau, Looker)
- CDC tools (Debezium, Fivetran, Qlik)
Hours: Overlap with US ET (start time 3:30 AM ET)
Job Types : Permanent, Contractual / Temporary
Work Location: Remote
📌 Databricks Architect (Remote) (India)
🏢 Sky Systems
📍 India