Azure Data Lead

Experience: Not Available to Not Available years

Location: Noida, India

Skills: Databricks, PySpark, SQL, Delta Lake, Spark, Azure Data Factory, Oracle PL/SQL, Python, data modeling, data warehousing, data governance, data security, data visualization

Job Summary
Design, develop, and maintain ETL/ELT data pipelines using Databricks (PySpark, SQL, Delta Lake). Build and optimize Spark-based data processing workflows for large-scale datasets. Develop and manage Databricks notebooks, jobs, and workflows. Integrate data from multiple structured and unstructured data sources. Implement and maintain Delta Lake tables, data models, and storage structures. Ensure data quality, performance optimization, security, and governance. Collaborate with data scientists, analysts, and business stakeholders to translate requirements into technical solutions. Troubleshoot and resolve performance, reliability, and data issues in Databricks environments. Follow DevOps and CI/CD best practices using tools like Git and Azure DevOps.

Key Responsibilities
1. Lead and manage end-to-end data engineering projects using azure data factory, azure databricks, sql, oracle pl/sql, and python.
2. Collaborate with stakeholders to gather and understand requirements for data pipelines and analytics solutions.
3. Design and develop etl processes,



data models, and data integration solutions.
4. Provide technical guidance and mentorship to the team members.
5. Ensure data quality, data governance, and data security standards are maintained throughout the project lifecycle.
6. Troubleshoot and optimize data pipelines and processes for performance and efficiency.
7. Stay updated on the latest trends and technologies in data engineering and contribute to continuous improvement efforts.

Skill Requirements
1. Proficiency in azure data factory (adf) and azure databricks for building and managing data pipelines.
2. Robust experience with sql and oracle pl/sql for data querying and manipulation.
3. Advanced programming skills in python for scripting and data processing tasks.
4. Knowledge of data modeling, data warehousing concepts, and database design principles.
5. Ability to work in a collaborative team environment and communicate effectively with stakeholders.
6. Strong analytical and problem-solving skills with attention to detail.
7. Experience in data visualization tools and techniques is a plus.

Other Requirements
Relevant certifications in Azure Data Factory, Azure Databricks, SQL, Oracle PL/SQL, or Python are advantageous.

📌 Azure Data Lead (Noida)
🏢 HCL Technologies
📍 Noida