Job Summary

DNI GSOF - DE Engineer

Must have Skills - Python, Spark, SQL, ADF, Synapse,

Databricks, Fabric, ADLS Gen2, Blob, ETL, CI/CD

Key Responsibilities

Responsibilities:

- Design, develop, and maintain data pipelines and ETL processes using Microsoft Azure services (e.g., Azure Data Factory, Azure Synapse, Azure Databricks, Azure Fabric).
- Utilize Azure data storage accounts for organizing and maintaining data pipeline outputs. (e.g., Azure Data Lake Storage Gen 2 & Azure Blob storage).
- Collaborate with data scientists, data analysts, data architects and other stakeholders to understand data requirements and deliver high-quality data solutions.
- Optimize data pipelines in the Azure environment for performance, scalability, and reliability.
- Ensure data quality and integrity through data validation techniques and frameworks.
- Develop and maintain documentation for data processes, configurations, and best practices.
- Monitor and troubleshoot data pipeline issues to ensure timely resolution.
- Stay current with industry trends and emerging technologies to ensure our data solutions remain cutting-edge.
- Manage the CI/CD process for deploying and maintaining data solutions.

Skill Requirements

Requirements:

- Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience)



and able to demonstrate high proficiency in programming fundamentals.
- Strong knowledge of Microsoft Azure services, including Azure Data Factory, Azure Synapse, Azure Databricks, Azure Blob Storage and Azure Data Lake Gen 2.
- Experience utilizing SQL DML to query modern RDBMS in an efficient manner (e.g., SQL Server, PostgreSQL).

Required Technical Skills

- Technical Knowledge:
- - Experience with Python, Spark & SQL.
- Experience with data transformation and orchestration tools such as Azure Data Factory, Databricks, Synapse, Azure Functions, Logic Apps, and others.
- Knowledge of structured, semi-structured and unstructured data sources.

- Teamwork:
- - Demonstrates the ability to effectively work in a team environment, align to stakeholders and business objectives.

- Communication:
- - Have excellent verbal and written communication skills and demonstrates the ability to effectively work asynchronously across time zones.

- Technical Leadership:
- - Demonstrates strong technical leadership through mentoring, guiding a team through technical challenges and ensuring best practices and standards are being followed.

Must Have Skill

Azure Blob Storage,Azure Data Factory (ADF),Azure Synapse Analytics,CI/CD,Databricks,Python

Valuable to Have Skill

Other Requirements

📌 Senior Technical Lead (India)
🏢 HCLTech
📍 India