02 Aug
|
Sampoorna Consultants
|
Mumbai
02 Aug
Sampoorna Consultants
Mumbai
Senior Data Platform and Integration Engineer - J50692
Role & responsibilities
Job Location: Juinagar, Navi Mumbai
Work Mode: Hybrid (3 days/week in office)
Job Responsibilities
- Collaborate with business stakeholders, product owners and leadership to translate business requirements into scalable data solutions.
- Define and evolve the target architecture for companies enterprise data platform spanning Azure, Snowflake, Databricks and SAP BTP.
- Establish an API-first integration strategy (REST, event-driven, microservices) and govern its adoption across teams.• Drive the Clean Core strategy, enabling agile side-by-side extensions through SAP BTP while protecting the SAP S/4HANA core.
- Design and govern a secure Data Marketplace with automated classification, lineage and access policies using Microsoft Purview and Snowflake Horizon.
- Define architectural standards, design patterns, and best practices for data ingestion, transformation, and distribution.
- Evaluate and recommend emerging technologies (Microsoft Fabric, Snowflake Cortex/Intelligence, Databricks Genie, RAG/AI) to advance the platform roadmap.
Hands On Experience
- Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, and distribution across cloud environments.
- Build and manage integrations between SAP and non-SAP enterprise systems and cloud data platforms (Azure Data Factory, Databricks, Snowflake).
- Implement real-time data streaming and Change Data Capture (CDC) mechanisms using Kafka, Azure Event Hub, or similar technologies.
- Develop API-based and event-driven integration solutions, including authentication (OAuth2, OIDC) and API gateway configuration.
- Design advanced data models (relational, star, snowflake schemas) optimized for analytical workloads in Snowflake and Databricks.
- Automate deployment and lifecycle management using CI/CD pipelines (Azure DevOps, GitHub Actions, or equivalent).
- Optimize pipeline performance, scalability,
and cost efficiency; troubleshoot and resolve production issues.
- Implement data governance, security controls, encryption and lineage tracking within the platform.
Qualification
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related field.
- 6+ years of qualified experience in data engineering, data platform architecture, or cloud-based integration — with a demonstrable mix of strategic design and hands-on implementation.
- Strong proficiency in SQL and Python; additional experience with R or Java is a plus.
- Deep hands-on experience with Microsoft Azure services (Data Lake, Data Factory, Databricks) and Snowflake architecture and optimization.• Proven expertise in ETL/ELT frameworks and tools, including batch, CDC and streaming patterns.
- Strong knowledge of API-driven architectures, REST APIs, event-driven messaging (Kafka, Event Hub, Service Bus), and authentication protocols (OAuth2, OIDC).
- Experience integrating SAP systems (S/4HANA, BTP, Datasphere) with cloud data platforms.
- Solid understanding of data governance, security, encryption and lineage principles.
- Experience with CI/CD practices and infrastructure-as-code.
- Strong analytical, communication, and stakeholder management skills; comfortable working in cross-functional, international teams using agile methodologies.
Preferred Skills & Experience:
- Experience with Microsoft Fabric and Power Platform / Power BI.
- Hands-on exposure to AI/ML capabilities on Snowflake (Cortex, Intelligence), Databricks (Genie), or SAP Joule.
- Relevant certifications in Azure, Snowflake or Databricks.
- Experience in regulated industries such as pharmaceuticals or healthcare
- Understanding of Pharma 4.0 (the pharmaceutical-industry analog to Industrie 4.0) — digitalization of GxP-regulated manufacturing and quality processes, including MES/LIMS integration, data integrity and connectivity between shop-floor (OT) and enterprise (IT) systems.
📌 Senior Data Platform and Integration Engineer (Mumbai)
🏢 Sampoorna Consultants
📍 Mumbai