This role is responsible for leveraging business intelligence, data analytics, data modeling, and big data technologies to transform complex data into actionable insights that support strategic decision-making. The individual will help improve business performance, optimize operations, identify market opportunities, and deliver data-driven solutions that create competitive advantage.

Additionally, the role ensures effective information management across the organization by supporting the creation, storage, governance, and maintenance of data throughout its lifecycle, enabling secure and efficient access, sharing, and use of information across teams, systems, and business functions.

Design and implement Lakehouse-based data architectures, including Medallion Architecture (Bronze, Silver, Gold layers), using configuration-driven and reusable frameworks to support multiple product teams.

Build, optimize, and maintain high-volume, high-velocity, and high-variety data pipelines for both batch and streaming workloads using Azure Databricks, Databricks, PySpark, and

SQL

.

Lead the design and development of data ingestion pipelines, data lake storage solutions, transformation notebooks, physical data models, tabular models, APIs/endpoints, and data visualization solutions.

Partner with business and product stakeholders to gather functional requirements, user stories, and product backlog items (PBIs), translating them into scalable technical solutions.

Define technical architecture, establish engineering standards, and provide technical direction across data engineering initiatives.

Develop and maintain reusable data engineering frameworks to improve scalability, consistency, and maintainability across projects.

Implement and support enterprise-grade data governance, quality, security, and performance optimization practices.

Collaborate with cross-functional teams, including Data Science, Analytics, and AI teams, to enable advanced analytics and AI-driven solutions.

Utilize Databricks Mosaic AI, LangGraph, or similar GenAI frameworks within enterprise guardrails to enhance data engineering workflows and accelerate solution delivery.

Mentor and guide junior engineers, conduct design reviews, and promote best practices in data engineering and software development.

Bachelor's Degree in Information Technology, Computer Science, Computer Engineering, Software Engineering, Data Science, Information Systems, or a related field.

With at least 8–12 years of experience in Data Engineering, Data Platform Development, Data Warehousing, or related data and analytics roles.

Strong hands-on experience with:

Databricks Mosaic AI

SQL

Transformation Notebooks

Proven experience designing and implementing scalable enterprise data platforms and large-scale data pipelines.

Experience with both batch and real-time/streaming data processing frameworks.

Robust knowledge of data modeling, ETL/ELT design patterns, and cloud-based data storage solutions.

Experience working with GenAI frameworks and platforms within enterprise environments and governance standards.

Experience in the Energy, Oil & Gas, Utilities, or related industry is highly preferred.

Required Certifications: Databricks Certified Data Engineer Professional (Mandatory)

You bring your skills and experience to Shell and in return you work with talented, committed people on one of the most important challenges facing our planet. Collaborate with experienced colleagues with unique expertise.

Benefit from flexible working hours, and the possibility of remote/mobile working.

Perform at your best with a competitive starting salary and annual performance related salary increase – our pay and benefits packages are considered to be among the best in the world.

Take advantage of paid parental leave, including for non-birthing parents.

We strongly encourage applicants of all genders, ages, ethnicities, cultures, abilities, sexual orientation, and life experiences to apply.

Grow as you progress through diverse career opportunities in national and

Gain access to a wide range of training and development programmes.

After you submit your application, you will receive an email providing a link to an online assessment that is a mandatory part of the process.

📌 Senior Data Engineer : Big Data (Bengaluru)
🏢 Shell
📍 Bengaluru