Role : Data Engineer IV

Location : remote

Contract : 6-12 Months

Interview : Video

Position Overview

We are seeking seven highly skilled Data Engineer IV professionals to lead the design, development, testing, and optimization of enterprise data solutions within the Azure ecosystem. This role requires deep expertise in Databricks, Azure Data Factory, Spark, SQL, and Power BI , along with strong leadership capabilities and the ability to work independently in a fast-paced Agile environment.

The ideal candidate will be responsible for building and supporting scalable data pipelines that move data from source systems into Azure Data Lake environments for reporting, analytics, and business intelligence. Candidates must have extensive experience working with Databricks, Unity Catalog, Delta Tables, and Power BI reporting solutions while ensuring data quality through effective unit and system testing practices.

All candidates must demonstrate strong communication skills, technical leadership experience, and the ability to guide engineering efforts across multiple workstreams.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using Databricks and Azure Data Factory.
- Develop Spark-based data transformation and processing solutions for large-scale data environments.
- Move and transform data from various source systems into Azure Data Lake environments.
- Utilize Databricks Unity Catalog and Delta Tables to manage and optimize enterprise data assets.
- Develop and support Power BI datasets, semantic models, reports, dashboards, and visualizations.
- Partner with business stakeholders to understand reporting and analytics requirements.
- Perform unit testing, integration testing, and system testing to ensure complete and accurate data processing.
- Validate that all required files and transactions are successfully captured and processed.
- Troubleshoot and resolve data pipeline and reporting issues.




- Participate in Agile ceremonies and contribute to sprint planning, backlog grooming, and delivery activities.
- Lead technical discussions, mentor team members, and promote engineering best practices.

Required Qualifications Must-Have Skills

· Databricks: 5+ years of experience o Databricks Certification required o Strong experience with Databricks architecture, optimization, and development

· Azure Data Factory: 5+ years of experience

· SQL: 5+ years of strong hands-on development experience

· Power BI Development: 3+ years of experience creating enterprise dashboards, reports, and data models

· Robust Spark programming experience

· Experience with Azure Data Lake environments

· Experience working with Unity Catalog and Delta Tables

· Strong understanding of data integration, transformation, and validation processes

· Experience performing unit testing and system testing of data solutions

· Strong verbal and written communication skills

· Strong understanding of Agile development methodologies

· Proven ability to function as a technical lead and mentor junior team members

Preferred Qualifications

· Experience within large enterprise data environments

· Azure cloud certifications

· Experience with CI/CD and DevOps practices for data engineering

· Knowledge of data governance, data quality, and security best practices

Leadership Expectations

· Lead and influence technical solution design and implementation.

· Provide guidance and mentorship to other engineers.

· Drive best practices related to data engineering, testing, and reporting.

· Communicate effectively with technical and business stakeholders.

· Take ownership of deliverables and provide proactive issue resolution.

Top Required Skills

- Databricks (5+ years) - Certification Required
- Azure Data Factory (5+ years)
- Strong SQL Development (5+ years)
- Power BI Development
- Spark Programming
- Unity Catalog & Delta Tables
- Azure Data Lake
- Agile Methodology
- Strong Communication Skills
- Technical Leadership Experience

📌 Data Engineer (India)
🏢 kpg99.INC
📍 India