Data Lead - Azure Data Factory (ADF), Databricks (India)

Data Lead - Azure Data Factory (ADF), Databricks (India)

05 Aug
|
HCLTech
|
India

05 Aug

HCLTech

India

Pune, Maharashtra
Job Summary

We are seeking a highly skilled Snowflake Engineer with 8+ years of overall IT experience, including 3+ years of strong hands-on experience in Snowflake-based data engineering. The ideal candidate brings deep expertise in modern data pipeline engineering, metadata-driven ingestion, and AI-enabled data processing, along with strong consultative and stakeholder engagement skills.

Key Responsibilities

Key Responsibilities

1. Data Pipeline Engineering

Design and develop scalable, high-performance data pipelines using SnowSQL, Snowpark, and Snowflake native capabilities
Build and optimize ELT/ETL frameworks supporting both batch and real-time workloads
Implement robust ingestion pipelines from diverse enterprise systems

2. Metadata-Driven Ingestion

Design and implement metadata-driven ingestion frameworks for scalable onboarding of datasets
Handle ingestion across multiple formats including XML, CSV, JSON, and Parquet
Automate ingestion, validation, and transformation processes

3. AI/GenAI Integration (Cortex AI)

Leverage Snowflake Cortex AI capabilities to enable AI-driven data transformations, enrichment, and insights generation
Collaborate with business and analytics teams to enable AI-powered data consumption use cases
Contribute to building intelligent, automated data pipelines and semantic layers

4. Data Modeling & Transformation

Develop scalable data models supporting analytics and reporting layers
Build transformations using Snowpark, SQL, and Snowflake native features
Optimize query performance and data access patterns

5. Data Delivery & Reporting Enablement

Deliver analytics-ready datasets in predefined formats for reporting tools (Power BI, Tableau, etc.)
Work closely with business stakeholders to ensure data usability and alignment with reporting requirements

6.



Performance Optimization & Governance

Implement best practices for performance tuning, clustering, and cost optimization
Ensure data quality, governance, security, and compliance standards
Monitor, troubleshoot, and continuously improve pipelines

7. Stakeholder Engagement & Technical Consulting

Provide technical consultation on data architecture, ingestion strategies, and Snowflake capabilities
Act as a trusted advisor in data engineering and platform decisions
Skill Requirements

Required Skills & Experience

Skill Category

Must-Have Skill

Expected Experience / Proficiency

Priority

Overall Experience

8+ years of overall IT experience

Strong background in data engineering, enterprise data platforms, and production delivery

Must Have

Snowflake Engineering

Snowflake hands-on engineering

Minimum 3+ years of experience with Snowflake platform implementation and support

Must Have

Snowflake Core Tools

SnowSQL, Snowpark, Snowpipe, Streams, and Tasks

Ability to design, build, automate, and operationalize Snowflake-native data pipelines

Must Have

Data Pipeline Engineering

ELT/ETL pipeline design and development

Experience building scalable batch and near real-time data pipelines

Must Have

Metadata-Driven Ingestion

Metadata-driven ingestion framework design

Ability to create reusable ingestion patterns for onboarding multiple datasets efficiently

Must Have

File Formats





Parquet, CSV, XML, and JSON

Hands-on experience handling structured and semi-structured data formats; Parquet is mandatory

Must Have

SQL & Data Modeling

Advanced SQL and scalable data modeling

Strong ability to write optimized SQL, design analytical models, and support reporting layers

Must Have

Snowflake Cortex AI

Hands-on experience or strong exposure to Snowflake Cortex AI

Ability to support AI-driven data transformation, enrichment, semantic search, or insight-generation use cases

Must Have

Performance Optimization

Snowflake performance tuning and cost optimization

Experience with query optimization, clustering strategy, warehouse sizing, and monitoring

Must Have

Governance & Security

Data quality, security, governance, and compliance

Ability to apply enterprise-grade controls, access patterns, data validation, and audit readiness

Must Have

Reporting Enablement

Delivery of analytics-ready datasets

Experience preparing curated datasets for BI/reporting tools such as Power BI, Tableau, or similar platforms

Must Have

Stakeholder Management

Communication, presentation, and consulting mindset

Ability to engage with business, analytics, and technology stakeholders and provide solution guidance

Must Have

Valuable-to-Have Skills

Experience with DBT (Data Build Tool)
Knowledge of PL/SQL or procedural SQL programming
Exposure to cloud platforms (AWS/Azure/GCP)
Familiarity with CI/CD pipelines and version control systems

#body.unify div.unify-button-container .unify-apply-now: focus, #body.unify div.unify-button-container .unify-apply-#body.unify div.unify-button-container .unify-apply-now: focus, #body.unify div.unify-button-container .unify-apply-

📌 Data Lead - Azure Data Factory (ADF), Databricks (India)
🏢 HCLTech
📍 India

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: data lead - azure data factory (adf), databricks (india) / india

Subscribe to this job alert:

Get the latest job offers by email for: data lead - azure data factory (adf), databricks (india) / india