Azure Databricks Engineer

Job Title

Azure Databricks Engineer – Design & Solutioning

Experience

7–12 Years

Job Location

Bengaluru

Employment Type

Full-Time

-

Job Summary

We are looking for an experienced Azure Databricks Engineer with 7–12 years of experience in designing, developing, and implementing scalable data engineering solutions on Microsoft Azure. The ideal candidate will have robust hands-on expertise in Azure Databricks, Azure Data Factory (ADF), data engineering, solution architecture, and technical design.

The candidate will be responsible for translating business and technical requirements into robust data solutions, participating in architecture and solutioning discussions, and leading the implementation of Azure-based data platforms.

Key Responsibilities

- Design and develop scalable and high-performance data engineering solutions using Azure Databricks.
- Design and implement data pipelines using Azure Data Factory (ADF).
- Develop complex ETL/ELT workflows using Databricks and Azure services.
- Work with stakeholders to understand business requirements and translate them into technical solutions.
- Participate in solution design, architecture, estimation, and technical discussions.
- Create technical design documents, data flow diagrams, solution architecture documents, and implementation plans.
- Optimize Databricks workloads, Spark jobs, notebooks, and data pipelines for performance and cost.
- Develop and maintain data transformation processes using Apache Spark/PySpark and SQL.
- Implement data ingestion and integration solutions across multiple data sources.
- Establish appropriate data processing,



storage, security, monitoring, and governance practices.
- Troubleshoot performance, data quality, pipeline failures, and production issues.
- Collaborate with data architects, cloud engineers, developers, business analysts, and other technical teams.
- Provide technical guidance and mentoring to junior and mid-level engineers.
- Ensure solutions follow Azure cloud best practices, security standards, and enterprise architecture principles.

Required Technical Skills

Azure

- Strong hands-on experience with Microsoft Azure.
- Good understanding of Azure data and analytics services.
- Experience designing cloud-based data solutions.

Azure Databricks

- Strong hands-on experience with Azure Databricks.
- Expertise in Apache Spark and PySpark.
- Strong SQL skills.
- Experience with Databricks notebooks, clusters, jobs, workflows, and performance optimization.
- Understanding of Delta Lake and modern data engineering architectures.

Azure Data Factory

- Strong experience developing and managing ADF pipelines.
- Experience with datasets, linked services, triggers, parameters, variables, integration runtimes, and monitoring.
- Ability to design robust and reusable ETL/ELT pipelines.

Design & Solutioning





- Strong understanding of solution architecture and technical design.
- Ability to convert business requirements into scalable technical solutions.
- Experience preparing HLD/LLD and technical design documentation.
- Strong understanding of data flow, integration patterns, scalability, performance, security, and reliability.

Preferred Skills

- Experience with Delta Lake / Lakehouse architecture.
- Experience with Azure Data Lake Storage (ADLS).
- Knowledge of Azure Synapse Analytics.
- Experience with CI/CD and DevOps practices for data engineering.
- Familiarity with Git and Azure DevOps.
- Knowledge of data governance, security, and access-control mechanisms.
- Experience working in Agile/Scrum environments.

Key Competencies

- Strong analytical and problem-solving skills.
- Excellent solution-design and architectural thinking.
- Ability to independently drive technical discussions.
- Strong communication and stakeholder-management skills.
- Ability to work across business and technology teams.
- Robust focus on scalability, performance, security, and maintainability.

Education

Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related discipline.

Ideal Candidate Profile

The ideal candidate is a hands-on Azure Databricks specialist who can operate beyond pure development, someone capable of understanding business requirements, designing the appropriate Azure data architecture, defining the solution, and guiding the engineering team through implementation.

Experience: 7–12 Years

📌 Azure Databricks Engineer (India)
🏢 Informica Solutions
📍 India