17 Aug
|
Topcoder
|
Noida
We are seeking a highly experienced Databricks Architect to design, lead, and implement scalable data platforms using the Databricks Lakehouse architecture. The role requires strong expertise in cloud-based data engineering, big data technologies, excellent communication skills, and the ability to work closely with business and technical stakeholders.
Key Responsibilities
- Lead end-to-end architecture design and implementation of Databricks-based data platforms
- Define data architecture standards, best practices, and governance frameworks using Delta Lake and Unity Catalog
- Architect scalable batch, streaming, and real-time data pipelines using Apache Spark (PySpark/Scala)
- Drive cloud data modernization initiatives on Azure / AWS / GCP
- Design secure, high-performance data ingestion, transformation, and consumption layers
- Optimize Databricks workloads for performance, cost, and scalability
- Oversee CI/CD automation for data pipelines using DevOps best practices
- Collaborate with business users, analytics teams,
and leadership to translate requirements into scalable data solutions
Required Skills & Expertise
- Databricks Lakehouse Architecture (Delta Lake, Unity Catalog, Clusters, Workspaces)
- Apache Spark (PySpark / Scala / Spark SQL) at enterprise scale
- Robust experience in Data Engineering & Solution Architecture
- Cloud platforms: Azure Databricks / AWS Databricks / GCP Databricks
- Data orchestration tools: Azure Data Factory, Airflow, or similar
- Data modeling and lakehouse design patterns
- Performance tuning, cost optimization, and security best practices
- Excellent SQL and Python skills
Good to Have
- Databricks MLflow, MLOps, and AI/ML integration exposure
- Power BI or other BI/analytics tool integrations
- Experience with Snowflake, Synapse, or similar platforms
- Databricks or Cloud Architect certifications
Experience & Qualifications
- 10–12 years of overall IT experience
- 6+ years in Big Data / Databricks / Spark-based solutions
📌 Databricks Architect (Noida)
🏢 Topcoder
📍 Noida