Position: Databricks Architect

Experience: 15+ years, with strong recent experience in Databricks/Data Architecture

Location: Remote

Employment Type: Full-time

Job Summary:

We are seeking a highly experienced Databricks Architect with 15+ years of overall IT experience to lead the architecture and modernization of enterprise-scale data platforms. The candidate will be responsible for designing scalable Databricks Lakehouse architectures, defining data engineering standards, driving cloud data modernization, and providing technical leadership to large data engineering teams.

The ideal candidate should have a strong combination of enterprise data architecture, Databricks, Apache Spark/PySpark, cloud platforms, data governance, and technical leadership.

Key Responsibilities:

- Define and own the enterprise data architecture and Databricks Lakehouse strategy.
- Design scalable, secure, highly available and cost-effective Databricks architectures.
- Lead large-scale data platform modernization and migration initiatives.
- Architect batch and real-time data processing solutions using Apache Spark, PySpark and Spark SQL.
- Design solutions using Delta Lake, Unity Catalog, Databricks Workflows/Jobs and Lakehouse architecture.
- Establish architecture standards for data ingestion, transformation, storage, processing, security and governance.
- Provide technical leadership to data engineers, architects and development teams.
- Review existing data platforms and recommend modernization strategies.
- Drive migration from legacy technologies such as Hadoop, Teradata, Oracle, Informatica or traditional data warehouses to Databricks.
- Design and implement data governance, security, access control, lineage and data quality frameworks.
- Define CI/CD, DevOps and Infrastructure-as-Code strategies for Databricks environments.
- Optimize Spark workloads, Databricks clusters, SQL warehouses and overall cloud costs.




- Collaborate with business stakeholders, enterprise architects, cloud teams and engineering leadership.
- Provide architectural guidance, conduct design reviews and ensure adherence to enterprise technology standards.
- Mentor senior engineers and architects and establish best practices across the organization.

Mandatory Technical Skills:

Databricks:

- Databricks Lakehouse Platform
- Delta Lake
- Unity Catalog
- Databricks Workflows/Jobs
- Databricks SQL
- Delta Live Tables / Lakeflow
- Databricks security and governance

Data Engineering:

- Apache Spark
- PySpark
- Spark SQL
- Python
- Advanced SQL
- ETL/ELT architecture
- Data modeling
- Batch and streaming data processing

Cloud: Strong experience with at least one:

- Microsoft Azure
- AWS
- GCP

Cloud & Data Ecosystem:

- Azure Data Factory / AWS Glue / equivalent data integration tools
- Kafka / Structured Streaming
- Cloud Data Lake / Object Storage
- Data Warehousing
- API and data integration
- Terraform or equivalent IaC tools
- Git and CI/CD

Architecture & Leadership Requirements:

- 15+ years of overall IT experience
- Strong experience in Data Architecture / Solution Architecture
- Proven experience designing enterprise-scale data platforms
- Solid stakeholder management and communication skills
- Experience leading large technical teams and architecture initiatives
- Ability to translate business requirements into scalable technical solutions
- Strong understanding of security, governance, performance and cost optimization
- Experience working with enterprise architecture frameworks and technology standards

Preferred Qualifications:

- Databricks certification
- Cloud architecture certification such as Azure/AWS/GCP
- Experience with MLflow / Databricks ML
- Experience with Power BI/Tableau
- Experience with Snowflake
- Experience with Hadoop ecosystem
- Experience with large-scale cloud migration and modernization programs

📌 Databricks Architect (Pune)
🏢 Quantum Integrators
📍 Pune