Roles And Responsibilities:
- Design, develop, test, deploy, and maintain Azure Data Factory (ADF) pipelines to extract, transform, and load data from various sources into Azure Storage solutions.
- Collaborate with cross-functional teams to understand business requirements and design ADF workflows that meet those needs.
- Develop complex SQL queries using Oracle ADF or SSIS to extract data from disparate systems.
- Integrate ADF with other Microsoft services such as Azure Databricks for big data processing.
- Troubleshoot issues related to ADF pipeline failures and optimize performance.
Job Requirements
- Azure Services: Proficiency with Azure Data Factory, Azure SQL Database, Azure Data Lake Storage, Azure Synapse Analytics, Azure Databricks, etc.
- Programming Languages:
Knowledge of languages like Python, SQL, or Scala, depending on the data processing tools used.
- Data Modeling: Understanding of data modeling concepts and practices.
- Big Data Technologies: Experience with big data processing frameworks such as Apache Spark.
- Data Warehousing: Knowledge of data warehousing concepts and technologies.
- Robust understanding of Azure storage solutions including Blob Storage, File Shares, Data Lake Storage etc.
- Experience working on large-scale data migration projects involving multiple stakeholders