18 Aug
|
Visioncraft
|
India
Job Summary:We are looking for an experienced Azure Data Engineer with strong hands-on expertise in designing and implementing cloud-based data solutions on Microsoft Azure. The ideal candidate should have experience with Azure data warehouses, NoSQL databases, hybrid data environments, Azure Data Factory, Databricks, and Azure streaming services.Key Responsibilities:- Design, develop, and implement scalable Azure cloud data solutions and data pipelines.- Develop and maintain cloud-based data warehouses, Azure databases, NoSQL databases, and hybrid data solutions.- Build robust ETL/ELT pipelines using Azure Data Factory (ADF).- Develop and manage ADF components including Azure Functions, Logic Apps, Triggers, and Integration Runtime (IR).- Design and implement data processing solutions using Azure Databricks.- Develop data transformation and processing workflows using PySpark and Scala.- Implement real-time and streaming data solutions using Azure Stream Analytics and Event Hubs.- Work with HDInsight and other Azure data services for distributed data processing.- Integrate data from multiple structured and unstructured sources into enterprise data platforms.- Implement data quality, validation, monitoring, and error-handling mechanisms across data pipelines.- Optimize data pipelines and processing workloads for performance, scalability,
and cost efficiency.- Collaborate with data architects, analysts, developers, and business stakeholders to understand data requirements.- Troubleshoot data pipeline failures and production issues and implement appropriate solutions.- Follow Azure security, governance, data management, and development best practices.- Contribute to the design and implementation of modern cloud and hybrid data architectures.Required Skills & Experience:- 5+ years of hands-on experience in data engineering and cloud data platforms.- Strong experience implementing Azure Cloud data warehouses.- Hands-on experience with Azure databases, NoSQL databases, and hybrid data scenarios.- Strong experience with Azure Data Factory (ADF).- Valuable knowledge of Azure Functions, Logic Apps, Triggers, and Integration Runtime (IR).- Strong hands-on experience with Azure Databricks.- Experience with PySpark and/or Scala for large-scale data processing.- Experience with Azure Stream Analytics and Event Hub.- Knowledge of HDInsight and distributed data processing.- Strong understanding of ETL/ELT processes, data integration, and data pipeline architecture.- Good understanding of cloud data security, governance, monitoring, and performance optimization. (ref:hirist.tech)
📌 Azure Data Engineer (India)
🏢 Visioncraft
📍 India