Job Title: Senior Data Engineer
Overview:
We are seeking a skilled and motivated Azure Databricks Data Engineer to join our agile team. The ideal candidate will have strong experience with Python, Spark programming, and expertise in building and optimizing data pipelines in Azure Databricks. You will play a pivotal role in leveraging Databricks workflows, Databricks Asset Bundles, and CI/CD pipelines using GitHub to deliver high-performance data solutions. A solid understanding of Data Warehousing and Data Mart architecture in Databricks is critical for success in this role. If youre passionate about data engineering, cloud technologies, and scalable data architecture, wed love to hear from you
Key Responsibilities:
Python and Spark Programming:
- Develop and maintain scalable data pipelines using Apache Spark within Azure Databricks/ Python/ Pyspark.
- Write optimized, high-performance Spark jobs to process large volumes of data efficiently.
- Utilize PySpark for distributed data processing, transformation, and aggregation tasks.

Databricks Workflows:




- Design and implement Databricks Workflows to automate data pipeline execution, orchestrating complex workflows and batch jobs.
- Set up task dependencies, triggers, and notifications to ensure smooth and reliable execution.
- Monitor, troubleshoot, and optimize Databricks workflows for optimal performance and minimal failures.

Databricks Asset Bundles:
- Create and manage reusable components such as Databricks Asset Bundles, including notebooks, libraries, and models.
- Share and reuse asset bundles across teams to increase efficiency and ensure consistency in development.

CI/CD for Databricks Artifacts using GitHub:
- Implement CI/CD pipelines using GitHub Actions for the continuous integration and deployment of Databricks notebooks, jobs, and libraries.
- Automate the testing, building, and deployment processes to ensure smooth, consistent code delivery across environments.
- Collaborate with teams to impleme

📌 Senior Data Engineer (Bengaluru)
🏢 ADM
📍 Bengaluru