Job Title: Databricks Solution Engineer

Location: Remote

Employment Type: W2 Contract (No C2C)

About the Role

We are looking for a highly experienced Solution Engineer with a strong background in building and scaling cloud-native data platforms. This role is Databricks-heavy , focusing on Unity Catalog, Medallion Architecture, Delta Lake, governance, and modern data engineering patterns. You will collaborate with cross-functional teams to translate business needs into well-architected solutions and guide delivery across enterprise environments.

Key Responsibilities

- Design and lead implementation of end-to-end Databricks Lakehouse Platforms using Delta Lake, Delta Live Tables, and MLflow.
- Architect Medallion Architecture (Bronze/Silver/Gold) for structured, semi-structured, and streaming workloads.
- Implement governed Lakehouse patterns using Unity Catalog for access control, lineage, data classification, and secure sharing.
- Build scalable ETL/ELT pipelines using Databricks Notebooks, Workflows, SQL Warehouses, and Spark-based transformations.
- Develop real-time streaming pipelines with Auto Loader, Structured Streaming, and event-driven platforms (Kafka, Kinesis, Pub/Sub).
- Integrate Databricks with cloud-native services such as AWS Glue, Azure Data Factory, and GCP Dataform .
- Define distributed integration patterns using REST APIs , microservices, and event-driven architectures.
- Enforce data governance, RBAC/ABAC, encryption, secret management , and compliance controls.
- Optimize Delta Lake tables, Spark workloads, and cluster configurations using Photon and autoscaling patterns.




- Drive cloud cost optimization across storage, compute, and workflow orchestration.
- Participate in architecture reviews, set standards, and support engineering teams throughout execution.
- Stay current on Databricks capabilities including Unity Catalog updates, Lakehouse Federation, serverless compute, and AI/ML features.

Required Qualifications

- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
- 10+ years of experience in enterprise software, cloud architecture, or data engineering roles.
- Solid hands-on experience with Databricks, Apache Spark, Delta Lake , and Lakehouse platform design.
- Experience implementing and administering Unity Catalog for governance, lineage, and fine-grained access control.
- Experience designing Medallion Architecture for analytics and engineering workloads.
- Hands-on experience with cloud platforms such as AWS, Azure, or GCP , including storage, compute, and networking services.
- Experience with streaming technologies such as Kafka, Kinesis, or Pub/Sub .
- Strong understanding of data modeling, workflow orchestration (Airflow, Databricks Workflows, dbt) , and pipeline automation.
- Familiarity with Scala-based Spark workloads in addition to PySpark and SQL pipelines.
- Skilled in performance tuning, Spark optimization, cluster policies, and cloud cost management .
- Excellent communication skills for technical leadership and stakeholder collaboration.
- Certifications in Databricks, AWS/GCP/Azure Solution Architecture, or TOGAF are a plus.

📌 Databricks Solution Engineer (Bengaluru)
🏢 Cloudtech-innovations
📍 Bengaluru