05 Sep
|
Vysystems
|
Pune
Position: Data Integration Engineer – Databricks
Experience: 8+ Years
Location: Bangalore / Pune
Interview Process: HR Discussion – F2F Mandatory
Job Summary
We are looking for an experienced Data Integration Engineer with strong expertise in Databricks, Azure Data Services, Spark, Python, SQL, Delta Lake, and enterprise application integrations . The ideal candidate should have experience working with business and IT stakeholders and integrating data from enterprise platforms such as Salesforce, SAP, and Oracle ERP .
Key Responsibilities
- Collaborate with business and IT stakeholders to understand requirements and translate them into scalable data integration solutions.
- Design and develop ETL/ELT pipelines for enterprise applications and cloud data platforms.
- Build and optimize Databricks pipelines using Python, SQL, Spark, and Delta Lake.
- Work with Delta Live Tables (DLT), Unity Catalog, and Delta Sharing .
- Develop and manage integrations using Fivetran, Azure Data Factory, and custom connectors .
- Integrate data from platforms such as Salesforce, SAP, Oracle ERP , and other enterprise applications.
- Ensure data quality, validation, traceability, and consistency across integrated systems.
- Manage CI/CD pipelines using Azure DevOps for data workflows and infrastructure.
- Work with architecture and security teams on data governance, security, and access control .
- Maintain technical documentation, data mappings, data flows, and system interface specifications.
- Support UAT activities from both functional and technical perspectives.
- Collaborate with geographically distributed teams and provide technical guidance to junior engineers.
- Identify opportunities to improve pipeline performance, scalability, reliability, and operational efficiency.
Required Skills
- 8+ years of experience in Data Engineering, Data Integration, or Analytics.
- Strong hands-on experience with Databricks, Apache Spark, Python, SQL, and Delta Lake .
- Experience with Delta Live Tables (DLT), Unity Catalog, and Delta Sharing .
- Hands-on experience with Fivetran connectors .
- Strong experience with Azure Data Factory, Azure Data Lake, Azure Key Vault, and Azure DevOps .
- Experience developing enterprise ETL/ELT and data integration pipelines .
- Knowledge of data modeling, data governance, security, and access control .
- Experience working with business-facing stakeholders and cross-functional teams.
- Strong communication and analytical/problem-solving skills.
- Knowledge of Agile and DevOps methodologies .
Good to Have
- Experience integrating Salesforce, SAP, Oracle ERP , or similar enterprise platforms.
- Understanding of ERP/CRM business processes such as Order-to-Cash, Procure-to-Pay, and Lead-to-Prospect .
- Experience with hybrid on-premises and cloud architectures .
- Knowledge of cloud platforms such as AWS or GCP in addition to Azure.
- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field. Master's degree is preferred.
📌 Data Engineer (Pune)
🏢 Vysystems
📍 Pune