09 Sep
|
Vysystems
|
Pune
Hi Team,
Greetings!
Please find below the requirement for the Data Integration Engineer – Databricks position.
Position: Data Integration Engineer – Databricks
Experience: 8+ Years
Location: Bangalore / Pune
Employment Type: C2H – 6 Months
Interview Process: HR Discussion + F2F Mandatory
Job Summary
We are looking for an experienced Data Integration Engineer with strong hands-on expertise in Databricks, Azure Data Services, Apache Spark, Python, SQL, and Delta Lake .
The ideal candidate should have experience designing and developing scalable ETL/ELT and data integration pipelines , working with enterprise applications, and collaborating with business and IT stakeholders.
Key Responsibilities
- Collaborate with business and IT stakeholders to understand requirements and develop scalable data integration solutions.
- Design, develop, and maintain ETL/ELT pipelines for enterprise applications and cloud data platforms.
- Build and optimize Databricks pipelines using Python, SQL, Spark, and Delta Lake.
- Work with Delta Live Tables (DLT), Unity Catalog, and Delta Sharing .
- Develop and manage integrations using Fivetran, Azure Data Factory, and custom connectors .
- Integrate data from enterprise platforms such as Salesforce, SAP, Oracle ERP , and similar systems.
- Ensure data quality, validation, traceability, and consistency across integrated systems.
- Manage CI/CD pipelines using Azure DevOps for data workflows and infrastructure.
- Collaborate with architecture and security teams on data governance, security, and access control .
- Maintain technical documentation,
data mappings, data flows, and interface specifications.
- Support UAT activities from both functional and technical perspectives.
- Provide technical guidance to junior engineers and collaborate with geographically distributed teams.
- Identify opportunities to improve pipeline performance, scalability, reliability, and operational efficiency.
Required Skills
- 8+ years of experience in Data Engineering, Data Integration, or Analytics .
- Strong hands-on experience with Databricks, Apache Spark, Python, SQL, and Delta Lake .
- Experience with Delta Live Tables (DLT), Unity Catalog, and Delta Sharing .
- Hands-on experience with Fivetran connectors .
- Strong experience with Azure Data Factory, Azure Data Lake, Azure Key Vault, and Azure DevOps .
- Experience developing enterprise ETL/ELT and data integration pipelines .
- Good understanding of data modeling, data governance, security, and access control .
- Experience working with business stakeholders and cross-functional teams.
- Solid communication, analytical, and problem-solving skills.
- Valuable understanding of Agile and DevOps methodologies .
Good to Have
- Experience integrating Salesforce, SAP, Oracle ERP , or similar enterprise platforms.
- Understanding of Order-to-Cash (O2C), Procure-to-Pay (P2P), and Lead-to-Opportunity business processes.
- Experience with hybrid on-premises and cloud architectures .
- Exposure to AWS or GCP in addition to Azure.
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or a related field.
📌 Integration Engineer (Pune)
🏢 Vysystems
📍 Pune