Senior Data Engineer (Databricks)

Mandatory Skills

Spark,Databricks Engineer,Kafka,Stream Processing,SQL Development,Python,Snowflake

Skill to Evaluate

Spark,Databricks Engineer,Snowflake SQL,Kafka,Stream Processing,SQL Development,Python,Snowflake

Experience

6 to 8 Years

Location

Bengaluru

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging modern cloud data ecosystems (Databricks?and?Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the?AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.

API Ingestion & Orchestration:?Design and implement robust data ingestion frameworks leveraging?Databricks APIs?and external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.

Real-time Processing:?Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling:?Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.

Technical Leadership:?Act as a core problem-solver for complex data bottlenecks, provide technical governance, and mentor engineering teams on data best practices.

AI Integration:?Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs, Machine Learning pipelines, and advanced analytics solutions.





Education Qualificaiton

B Tech/ M Tech

Job Title

Senior Data Engineer (Databricks)

Roles & Responsibilities

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging modern cloud data ecosystems (Databricks?and?Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the?AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.

API Ingestion & Orchestration:?Design and implement robust data ingestion frameworks leveraging?Databricks APIs?and external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.

Real-time Processing:?Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling:?Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.

AI Integration:?Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs,



Machine Learning pipelines, and advanced analytics solutions.

Type of Employment

Contract

Project Details

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data architectures leveraging contemporary cloud data ecosystems (Databricks?and?Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components natively within the?AWS ecosystem, ensuring tight integration with core security, IAM, and networking protocols.

API Ingestion & Orchestration:?Design and implement robust data ingestion frameworks leveraging?Databricks APIs?and external REST/GraphQL APIs for automated workflows, platform orchestration, and data delivery.

Real-time Processing:?Design and implement robust frameworks for real-time data ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling:?Harmonize "old school" relational data warehousing patterns (Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern paradigms.

Technical Leadership:?Act as a core problem-solver for complex data bottlenecks, provide technical governance, and mentor engineering teams on data best practices.

AI Integration:?Collaborate with Data Science and AI teams to architect data layers that seamlessly support LLMs, Machine Learning pipelines, and advanced analytics solutions.

Project Duration

12

Shift Timings

General

Kafka, Databricks, Python, Snowflake

📌 SENIOR DATA ENGINEER - Databricks (Bengaluru)
🏢 Happiest Minds Technologies
📍 Bengaluru