Job Summary

We are looking for a Senior Data Engineer skilled in Databricks and Big Data technologies. You will focus on creating scalable data architectures, pipelines, and real-time streaming frameworks using Databricks, Apache Spark, and Kafka, preferably on Azure. Your role will ensure data governance, security, and high performance in a cloud setting. Location: Bangalore. Employment type: Full time.

Responsibilities
Develop end-to-end data solutions using Databricks for large-scale processing.
Build and optimize data pipelines for high quality, reliability, and scalability.
Perform complex data transformations with PySpark and Databricks.
Integrate Kafka for real-time data streaming and processing.
Collaborate with DevOps to optimize Databricks clusters and Kafka setups.
Implement best practices for data governance, security, and privacy.
Troubleshoot pipelines, resolve performance issues, and ensure data quality.
Plan and scale Databricks and Kafka infrastructures for large workloads.
Document workflows, processes, and configurations.
Work with relational databases (e.g., MySQL, MS SQL) and understand complex SQL procedures.





Requirements
3+ years experience in Big Data tools: Databricks, Apache Spark, Kafka, and Cloud platforms.
Proficiency in programming: Python, PySpark, Scala, or Java.
Strong SQL skills with relational/NoSQL databases.
Experience with data modeling, data lakes, and scalable solutions.
Knowledge of Azure (preferred) or other clouds like AWS/GCP.
Expertise in data governance, security, and quality in Databricks.
Familiar with Docker, Linux scripting, and Agile workflows.
Certifications in Databricks or Azure Data Engineering.
Experience with additional tools like Snowflake.
Robust documentation, communication, and stakeholder management skills.

Skills
Databricks
Apache Spark
Kafka
PySpark
Python
SQL
Azure
Scala
Java

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Senior Data Engineer Udaipur (India)
🏢 WebSenor
📍 India