11 Sep
|
Vysystems
|
Maharashtra
11 Sep
Vysystems
Maharashtra
Hi Team,
Greetings!
Please find below the requirement for the Data Integration Engineer – Databricks position.
Position: Data Integration Engineer – Databricks
Experience: 8+ Years
Location: Bangalore / Pune
Employment Type: C2H – 6 Months
Interview Process: HR Discussion + F2F Mandatory
Job Summary
We are looking for an experienced Data Integration Engineer with strong hands-on expertise in Databricks, Azure Data Services, Apache Spark, Python, SQL, and Delta Lake.
The ideal candidate should have experience designing and developing scalable ETL/ELT and data integration pipelines, working with enterprise applications, and collaborating with business and IT stakeholders.
Key Responsibilities
- Collaborate with business and IT stakeholders to understand requirements and develop scalable data integration solutions.
- Design, develop, and maintain ETL/ELT pipelines for enterprise applications and cloud data platforms.
- Build and optimize Databricks pipelines using Python, SQL, Spark, and Delta Lake.
- Work with Delta Live Tables (DLT), Unity Catalog, and Delta Sharing.
- Develop and manage integrations using Fivetran, Azure Data Factory, and custom connectors.
- Integrate data from enterprise platforms such as Salesforce, SAP, Oracle ERP, and similar systems.
- Ensure data quality, validation, traceability, and consistency across integrated systems.
- Manage CI/CD pipelines using Azure DevOps for data workflows and infrastructure.
- Collaborate with architecture and security teams on data governance, security, and access control.
- Maintain technical documentation,
data mappings, data flows, and interface specifications.
- Support UAT activities from both functional and technical perspectives.
- Provide technical guidance to junior engineers and collaborate with geographically distributed teams.
- Identify opportunities to improve pipeline performance, scalability, reliability, and operational efficiency.
Required Skills
- 8+ years of experience in Data Engineering, Data Integration, or Analytics.
- Solid hands-on experience with Databricks, Apache Spark, Python, SQL, and Delta Lake.
- Experience with Delta Live Tables (DLT), Unity Catalog, and Delta Sharing.
- Hands-on experience with Fivetran connectors.
- Strong experience with Azure Data Factory, Azure Data Lake, Azure Key Vault, and Azure DevOps.
- Experience developing enterprise ETL/ELT and data integration pipelines.
- Good understanding of data modeling, data governance, security, and access control.
- Experience working with business stakeholders and cross-functional teams.
- Strong communication, analytical, and problem-solving skills.
- Good understanding of Agile and DevOps methodologies.
Good to Have
- Experience integrating Salesforce, SAP, Oracle ERP, or similar enterprise platforms.
- Understanding of Order-to-Cash (O2C), Procure-to-Pay (P2P), and Lead-to-Opportunity business processes.
- Experience with hybrid on-premises and cloud architectures.
- Exposure to AWS or GCP in addition to Azure.
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or a related field.
📌 Integration Engineer (Maharashtra)
🏢 Vysystems
📍 Maharashtra