We are seeking a highly skilled Azure Data Engineer with strong expertise in designing, developing, and optimizing scalable data solutions on the Microsoft Azure platform. The ideal candidate should have hands-on experience in Azure Data Factory (ADF), Azure Databricks, PySpark, Python, SQL Server, and contemporary data warehousing concepts.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build and optimize data processing frameworks using PySpark and Python.
- Develop and maintain data models, data transformations, and integration processes.
- Work with SQL Server databases for data extraction, transformation, and loading activities.
- Implement data quality, monitoring, and performance tuning for large-scale datasets.
- Collaborate with business stakeholders, analysts, and development teams to understand data requirements.
- Support data migration projects from on-premises systems to Azure cloud platforms.
- Create and maintain technical documentation and best practices for data engineering solutions.
- Ensure compliance with data governance, security, and regulatory standards.
Required Skills
- Strong experience in Azure Data Engineering (ADE).
- Hands-on experience with Azure Data Factory (ADF).
- Expertise in Azure Databricks and Databricks ecosystem.
- Strong programming skills in Python and PySpark.
- Advanced knowledge of SQL Server and complex SQL queries.
- Experience with ETL/ELT design and implementation.
- Understanding of Data Warehousing and Data Lake concepts.
- Experience in data pipeline optimization and performance tuning.
- Knowledge of Azure Cloud services and architecture.
Preferred Skills
- Experience with Azure Synapse Analytics.
- Knowledge of Delta Lake and Lakehouse Architecture.
- Familiarity with CI/CD pipelines using Azure DevOps.
- Experience with Git version control.
- Understanding of Agile/Scrum methodologies.
Qualifications
- Bachelor's/Master's degree in Computer Science, Information Technology, or related field.
- 5+ years of experience in Data Engineering.
- Relevant Microsoft Azure certifications are preferred.
Mandatory Skills
📌 Data Analytics Engineer (Hyderabad)
🏢 HCLTech
📍 Hyderabad
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.