Databricks Data Specialist707 (Bengaluru)

Databricks Data Specialist707 (Bengaluru)

16 Nov
|
brillio-2
|
Bengaluru

16 Nov

brillio-2

Bengaluru

Data Specialist
Primary Skills

Databricks Engineer

Role Overview

We are seeking a highly skilledDatabricks Engineerto design, develop, and optimize scalable data engineering and analytics solutions on theDatabricks Lakehouse Platform. The ideal candidate will possess strong expertise inDatabricks, PySpark, and SQL, with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.
Key Responsibilities

Design, develop, and maintain end-to-end data pipelines usingDatabricksandPySpark.
Build and implementLakehouse architecturesutilizing Bronze, Silver, and Gold data layers.
Develop and manageDelta Lakesolutions with ACID transactions, schema enforcement, and data reliability features.
Create, monitor, and optimizeDelta Live Tables (DLT)pipelines.
Implement scalable and efficient data ingestion processes usingAuto Loader.
Develop and manage real-time data processing solutions usingStructured Streaming.
Orchestrate, schedule, and monitor data workflows usingDatabricks Workflows.
Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
Establish and enforce data governance, security, and access controls usingUnity Catalog.
Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.

Required Skills (Must Have)

Databricks PlatformDelta Lake
Delta Live Tables (DLT)
Unity Catalog
Databricks Workflows

PySpark and Apache Spark
Structured Streaming
Auto Loader
SQL
Lakehouse Data Modeling
Strong understanding of data engineering best practices and scalable data architectures

Preferred Skills (Valuable to Have)

Azure Ecosystem

Azure Data Factory (ADF)
Azure Synapse Analytics
Microsoft Purview
Microsoft Fabric

AWS Ecosystem

AWS Glue
AWS Lambda
AWS Step Functions

Data Engineering & Integration

Apache Airflow
DBT
Fivetran
Informatica

Streaming & Analytics

Apache Kafka
Power BI

Data Governance

Collibra
Alation

GCP

BigQuery

Qualifications

Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
Strong analytical, troubleshooting,



and problem-solving capabilities.
Experience working in agile and collaborative environments.
Excellent communication and stakeholder management skills.

Preferred Candidate Profile

Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
Strong understanding of data governance, security, and compliance frameworks.
Experience delivering both batch and real-time data processing solutions.
Ability to work independently while collaborating effectively across global teams.

Key Technologies

Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI
Specialization

Databricks Engineering: Lead Data Engineer
Job requirements

Databricks Engineer

Role Overview

We are seeking a highly skilledDatabricks Engineerto design, develop, and optimize scalable data engineering and analytics solutions on theDatabricks Lakehouse Platform. The ideal candidate will possess strong expertise inDatabricks, PySpark, and SQL, with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.
Key Responsibilities

Design, develop, and maintain end-to-end data pipelines usingDatabricksandPySpark.
Build and implementLakehouse architecturesutilizing Bronze, Silver, and Gold data layers.
Develop and manageDelta Lakesolutions with ACID transactions, schema enforcement, and data reliability features.
Create, monitor, and optimizeDelta Live Tables (DLT)pipelines.
Implement scalable and efficient data ingestion processes usingAuto Loader.
Develop and manage real-time data processing solutions usingStructured Streaming.
Orchestrate, schedule, and monitor data workflows usingDatabricks Workflows.
Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
Establish and enforce data governance, security, and access controls usingUnity Catalog.
Optimize Spark jobs,



SQL queries, and overall platform performance to ensure efficiency and scalability.
Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.

Required Skills (Must Have)

Databricks PlatformDelta Lake
Delta Live Tables (DLT)
Unity Catalog
Databricks Workflows

PySpark and Apache Spark
Structured Streaming
Auto Loader
SQL
Lakehouse Data Modeling
Strong understanding of data engineering best practices and scalable data architectures

Preferred Skills (Good to Have)

Azure Ecosystem

Azure Data Factory (ADF)
Azure Synapse Analytics
Microsoft Purview
Microsoft Fabric

AWS Ecosystem

AWS Glue
AWS Lambda
AWS Step Functions

Data Engineering & Integration

Apache Airflow
DBT
Fivetran
Informatica

Streaming & Analytics

Apache Kafka
Power BI

Data Governance

Collibra
Alation

GCP

BigQuery

Qualifications

Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
Strong analytical, troubleshooting, and problem-solving capabilities.
Experience working in agile and collaborative environments.
Excellent communication and stakeholder management skills.

Preferred Candidate Profile

Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
Strong understanding of data governance, security, and compliance frameworks.
Experience delivering both batch and real-time data processing solutions.
Ability to work independently while collaborating effectively across global teams.

Key Technologies

Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

📌 Databricks Data Specialist707 (Bengaluru)
🏢 brillio-2
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: databricks data specialist707 (bengaluru) / bengaluru