We are looking for an experienced Senior Data Engineer to design, develop, and maintain scalable data pipelines and analytics solutions using Microsoft Azure and Databricks. The ideal candidate will have strong expertise in Python, SQL, real-time data processing, and data architecture, along with experience leading technical decisions and collaborating directly with business and client stakeholders.

Key Responsibilities

- Own architecture and technical design decisions for client data platforms and communicate solutions directly to clients.
- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build robust ETL/ELT processes to ingest, transform, and load data from multiple sources.
- Develop clean, efficient, and reusable Python code for data processing and automation.
- Optimize SQL queries for data extraction, transformation, and reporting.
- Implement and manage Delta Lake and Unity Catalog solutions.
- Develop and support real-time data processing solutions using frameworks such as Spark Streaming and Kafka.
- Establish engineering standards for code reviews, testing, version control, and CI/CD practices.
- Ensure data quality, integrity, security, availability, and performance across data platforms.
- Monitor and troubleshoot production data pipelines, implementing logging, alerting, and error-handling mechanisms.
- Collaborate with business stakeholders to understand requirements and deliver solutions aligned with business objectives.




- Develop data models and interactive dashboards using Power BI, including DAX.
- Mentor junior data engineers and provide technical guidance to the team.

Required Skills and Qualifications

- 8+ years of experience in Data Engineering or related roles.
- Strong hands-on experience with Microsoft Azure data services, particularly:
- Azure Data Factory (ADF)
- Azure Databricks
- Delta Lake
- Unity Catalog
- Strong programming skills in Python.
- Advanced SQL skills and a solid understanding of relational databases.
- Production experience with real-time data processing frameworks such as Spark Streaming and Kafka.
- Experience maintaining and supporting production-grade data pipelines.
- Hands-on experience with Power BI, data modelling, DAX, and interactive dashboard development.
- Experience with Git or other version control systems and CI/CD pipelines.
- Proven ability to own technical architecture and design decisions.
- Solid communication and collaboration skills, with the ability to work directly with clients and business stakeholders.
- Excellent problem-solving skills and attention to detail.

Good to Have

- Experience with ETRM/CTRM platforms, energy trading, or commodity trading systems.
- Understanding of market data, trade lifecycle, and financial instruments.
- Databricks certifications, such as Data Engineer Associate or Professional.
- Current Microsoft Azure Data Engineering certification.

📌 Senior Data Engineer (Bengaluru)
🏢 360 Technology
📍 Bengaluru