06 Sep
|
Quantum Integrators
|
Pune
06 Sep
Quantum Integrators
Pune
Position: Databricks Architect
Experience: 15+ years, with robust recent experience in Databricks/Data Architecture
Location: Remote
Employment Type: Full-time
Job Summary:
We are seeking a highly experienced Databricks Architect with 15+ years of overall IT experience to lead the architecture and modernization of enterprise-scale data platforms. The candidate will be responsible for designing scalable Databricks Lakehouse architectures, defining data engineering standards, driving cloud data modernization, and providing technical leadership to large data engineering teams.
The ideal candidate should have a strong combination of enterprise data architecture, Databricks, Apache Spark/PySpark, cloud platforms, data governance, and technical leadership.
Key Responsibilities:
Define and own the enterprise data architecture and Databricks Lakehouse strategy.
Design scalable, secure, highly available and cost-effective Databricks architectures.
Lead large-scale data platform modernization and migration initiatives.
Architect batch and real-time data processing solutions using Apache Spark, PySpark and Spark SQL.
Design solutions using Delta Lake, Unity Catalog, Databricks Workflows/Jobs and Lakehouse architecture.
Establish architecture standards for data ingestion, transformation, storage, processing, security and governance.
Provide technical leadership to data engineers, architects and development teams.
Review existing data platforms and recommend modernization strategies.
Drive migration from legacy technologies such as Hadoop, Teradata, Oracle, Informatica or traditional data warehouses to Databricks.
Design and implement data governance, security, access control, lineage and data quality frameworks.
Define CI/CD, DevOps and Infrastructure-as-Code strategies for Databricks environments.
Optimize Spark workloads, Databricks clusters, SQL warehouses and overall cloud costs.
Collaborate with business stakeholders, enterprise architects, cloud teams and engineering lead
📌 Databricks Architect (Pune)
🏢 Quantum Integrators
📍 Pune