What’s the role?
This role is responsible for leveraging business intelligence, data analytics, data modeling, and big data technologies to transform complex data into actionable insights that support strategic decision-making. The individual will help improve business performance, optimize operations, identify market opportunities, and deliver data-driven solutions that create competitive advantage.
Additionally, the role ensures effective information management across the organization by supporting the creation, storage, governance, and maintenance of data throughout its lifecycle, enabling secure and effective access, sharing, and use of information across teams, systems, and business functions.
What you’ll be doing?
• Design and implement Lakehouse-based data architectures, including Medallion Architecture (Bronze, Silver, Gold layers), using configuration-driven and reusable frameworks to support multiple product teams.
• Build, optimize, and maintain high-volume, high-velocity, and high-variety data pipelines for both batch and streaming workloads using Azure Databricks,
Databricks, PySpark, and SQL.
• Lead the design and development of data ingestion pipelines, data lake storage solutions, transformation notebooks, physical data models, tabular models, APIs/endpoints, and data visualization solutions.
• Partner with business and product stakeholders to gather functional requirements, user stories, and product backlog items (PBIs), translating them into scalable technical solutions.
• Define technical architecture, establish engineering standards, and provide technical direction across data engineering initiatives.
• Develop and maintain reusable data engineering frameworks to improve scalability, consistency, and maintainability across projects.
• Implement and support enterprise-grade data governance, quality, security, and performance optimization practices.
• Collaborate with cross-functional teams, including Data Science, Analytics, and AI teams, to e
📌 Senior Data Engineer (India)
🏢 Shell
📍 India