Role: Data Pipeline Engineer

- Overall experience- 8+ years.
- Shift Timings- 3.30pm to 1.30am IST
- Work Mode-WFO (5 Days)

Key Responsibilities

Design, build, and maintain end-to-end data pipelines on Azure using modern data engineering patterns (batch, streaming, ELT/ETL, event-driven ingestion).

Implement data ingestion from diverse sources, including APIs, databases, files, event streams, and SaaS platforms.

Develop scalable data transformation and enrichment processes to support analytics, reporting, and AI/ML workloads.

Design and manage data storage solutions across lakehouse, warehouse, and operational data stores. Ensure data quality, consistency, lineage, and observability through validation, monitoring, logging, and alerting.

Optimize pipeline performance, reliability, and cost through tuning, partitioning, scheduling, and resource management.

Integrate data pipelines with downstream systems, including BI tools, AI/RAG workloads, and operational applications.

Collaborate with security and governance teams to implement access controls, data protection, and compliance requirements.

Support CI/CD practices for data pipelines, including versioning, automated testing, and environment promotion.

Required Qualifications

5+ years of skilled experience designing and building production data pipelines.

Hands-on experience with Azure data and integration services, such as: o Azure Data Factory / Synapse Pipelines o Azure Functions and/or Logic Apps o Azure Storage (ADLS Gen2, Blob) o Azure SQL, Synapse Analytics, or Cosmos DB

Strong experience with data transformation using SQL and at least one programming language such as Python or C#.

Experience working with structured, semi-structured, and unstructured data formats (CSV, JSON, Parquet, etc.).

Familiarity with batch and streaming data processing concepts.

Experience building integrations with REST APIs and external data sources.

Understanding of DevOps practices, including Git workflows, CI/CD pipelines, and infrastructure-as-code.





Strong problem-solving skills and the ability to translate business and data requirements into robust technical solutions.

Preferred Qualifications

Experience supporting AI, ML, or RAG workloads with curated, searchable, and well structured data.

Familiarity with event streaming technologies and patterns (e.g., Event Hubs, Kafka-like concepts).

Experience designing lakehouse or medallion-style data architectures.

Knowledge of data quality frameworks, data cataloging, and lineage tools.

Exposure to containerized or distributed data processing workloads (e.g., Spark-based systems).

Core Technologies

Azure Data Factory / Synapse Pipelines

Azure Functions / Logic Apps

Azure Data Lake Storage Gen2 / Blob Storage

Azure SQL, Synapse Analytics, Cosmos DB

Azure Event Hubs

Azure DevOps

VDart: Digital Consulting & Staffing Solutions

VDart is a leading digital consulting and staffing company founded in 2007 and headquartered in Alpharetta, Georgia. VDart is one of the top staffing companies in USA & also provides technology solutions, supporting businesses in their digital transformation journey. Digital consulting services, Digital consulting agency

Core Services:

- Digital consulting and transformation solutions
- Comprehensive staffing services (contract, temporary, permanent placements)
- Talent management and workforce solutions
- Technology implementation services

Key Industries: VDart primarily serves industries such as automotive and mobility, banking and finance, healthcare and life sciences, and energy and utilities. VDart - Products, Competitors, Financials, Employees, Headquarters Locations

Notable Partnership: VDart Digital has been a consistently strong supplier for Toyota since 2017 Digital Transformation Consulting & Strategy Services | VDart Digital, demonstrating their capability in large-scale digital transformation projects.

With over 17 years of experience, VDart combines staffing expertise with technology solutions to help organizations modernize their operations and build skilled workforce capabilities across multiple sectors.

📌 Data Pipeline Engineer (Bengaluru)
🏢 VDart Software Services
📍 Bengaluru