We are looking for a highly skilled Data Engineer with hands-on experience in Azure Databricks, Azure Data Factory (ADF), Delta Lake, and metadata-driven frameworks. The ideal candidate will have a strong background in building scalable data pipelines and solutions in the banking or financial services domain, with a focus on performance, governance, and data quality.

Key Responsibilities

Design, develop, and maintain scalable and efficient data pipelines using Azure Databricks and ADF.

Implement metadata-driven frameworks to automate and standardize data ingestion, transformation, and orchestration processes.

Work with Delta Lake to manage large-scale data lakes with ACID transactions and time travel capabilities.

Collaborate with data architects, analysts, and business stakeholders to understand data requirements and deliver robust solutions.

Ensure data quality, lineage, and governance across all data engineering processes.

Optimize data workflows for performance, scalability, and cost-efficiency in the Azure cloud environment.

Support data integration from various banking systems and ensure compliance with regulatory and security standards.

Document technical designs,



data flows, and operational procedures.

Required Skills And Qualifications

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

4+ years of experience in data engineering, with at least 2 years in the Azure ecosystem.

Robust expertise in Azure Databricks, ADF, and Delta Lake.

Proven experience building metadata-driven data pipelines and frameworks.

Proficiency in PySpark, SQL, and data modeling techniques.

Solid understanding of banking data structures, products, and regulatory requirements.

Experience with CI/CD pipelines, DevOps practices, and Git.

Familiarity with data governance, data quality tools, and data cataloging (e.g., Azure Purview).

Preferred Qualifications

Experience with Azure Synapse Analytics, Azure Data Lake Storage Gen2, and Power BI.

Knowledge of data security best practices in financial services.

Exposure to real-time data processing using tools like Kafka or Event Hubs.

Certification in Azure Data Engineer Associate or related credentials.

📌 Senior Consultant (Hyderabad)
🏢 Virtusa
📍 Hyderabad