Databricks Engineer

Job Summary

The Sr. Database Engineer ETL specializes in engineering data movement in the acquisition and processing environment. The role maintains databases for executing Extract, Transform, and Load (ETL) processes for managing job-related data projects. The Database Engineer ETL will create processes and maintain data quality, security, integration, and perform data analysis operating with large volume data.

Mandatory Skills:

Azure Databricks(Min 5 + Years of experience), ETL , Pyspark, Azure Data factory , Data Lake, Synapse .

Job Responsibilities

Serve as a Subject Matter Expert in managing and developing governance, process and procedures for the database ETL environment.

Resolve complex problems and develop and implement solutions where expertise is required to interpret against policies, guidelines or processes.

Collaborate, consult, and present technical information to senior leadership as it pertains to the database ETL environment.

Typically referred to as a technical expert and key advisor within the database ETL environment Directs the workflow of other Database Engineer ETLs and mentors IT staff members

- Workplace Management

- Leads the design of data processes in Synapse, Databricks, SQL, and Spark that land, clean and transform data for the ODS
- Ensure the data processes are designed and tuned for optimal performance and make recommendations for change
- Operate with large data volumes and tune for best performance on data loading and transformation
- Design detailed documentation for lake house data structures and ensure accuracy is maintained
- Serves as a subject matter expert for data consumers to ensure a data driven culture exists

- ETL/ELT Processes

- Lead the design, creation, and management of ETL pipelines to extract job-related data from various sources, transforming it into a consistent format, and loads into the database.




- Lead the automation of data-related tasks, including data extraction, transformation, and loading ETL processes.
- Create and implement consistent standards across all workspaces for pipelines, Lakehouse structures, access rules, linked services, and compute pools.
- Ensure the integrity and design of the code repository Promote code repository within the defined environments.
- Designs and develops pipelines that scrape web data and Application Programming Interface (API) Translates highly customized SQL processes in stored procedures and SSIS into modern technology (Synapse/Databricks/SQL/SPARK).
- Utilize Synapse/Spark/SQL/Databricks at high level

- Data Quality

- Develop and maintain procedures to ensure data quality and consistency within the data repositories through data cleaning and validation
- Collaborate with all levels of the organization including senior management, Data Engineers and Data Scientists to ensure data quality, reliability, and accessibility

- Integration

Coordinate the integration of the data store with other systems, such as job boards, business software, or analytics tools.

- Security

- Implement security measures to protect data and the data repository including encryption and access control
- Utilize Azure data security and compliance measures to ensure data protection and adherence to regulations.
- Utilize security measures for data, including encryption and access control. Implement controls for libraries and code

- Data Analysis





- Designs and develops data analysis concepts and reporting as it relates to business purposes Appropriately assesses risk when business decisions are made, including but not limited to compliance and operational risk.
- Demonstrates consideration for client's reputation as well as our clients, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues.

VDart: Digital Consulting & Staffing Solutions

VDart is a leading digital consulting and staffing company founded in 2007 and headquartered in Alpharetta, Georgia. VDart is one of the top staffing companies in USA & also provides technology solutions, supporting businesses in their digital transformation journey. Digital consulting services, Digital consulting agency

Core Services:

- Digital consulting and transformation solutions
- Comprehensive staffing services (contract, temporary, permanent placements)
- Talent management and workforce solutions
- Technology implementation services

Key Industries: VDart primarily serves industries such as automotive and mobility, banking and finance, healthcare and life sciences, and energy and utilities. VDart - Products, Competitors, Financials, Employees, Headquarters Locations

Notable Partnership: VDart Digital has been a consistently strong supplier for Toyota since 2017 Digital Transformation Consulting & Strategy Services | VDart Digital, demonstrating their capability in large-scale digital transformation projects.

With over 17 years of experience, VDart combines staffing expertise with technology solutions to help organizations modernize their operations and build skilled workforce capabilities across multiple sectors.

📌 Databricks Engineer (Bengaluru)
🏢 VDart Software Services
📍 Bengaluru