We are looking for an experienced Azure Data Engineer to design, build, and support end-to-end data pipelines on the Azure ecosystem. The role involves extracting and transforming data from multiple source systems, building robust ADF/Synapse/Databricks pipelines, and supporting business-critical reporting and reconciliation processes.
Skills Required
Azure Data Factory (ADF)
Azure Databricks (ADB)
Azure Active Directory
Azure Storage
Azure Data Lake Services (ADLS)
Azure Key Vault
Azure Logic Apps
Azure SQL DB
Azure HDInsight
Good knowledge of Azure DevOps (ADO) services Repos, Boards, Build Pipelines (CI/CD)
Valuable knowledge of developing data engineering frameworks and notebooks using Azure Databricks with Spark SQL, PySpark, and Python
Positive knowledge of Azure Synapse Analytics
Positive-to-Have Skills
GitHub
PowerApps
Azure Monitor and Log Analytics
Power BI
REST APIs
Daily Activities
Extracting data from source systems and files in any format from SFTP locations.
Processing the extracted data and storing it into staging tables in Azure Blob Storage.
Transforming data based on the mapping (STTM) document to meet given requirements in ADB using SQL, Python,
and PySpark code.
Generating a final TLM file or report and loading it into the Smartstream environment.
Building pipelines end-to-end using ADF and Synapse.
Monitoring pipelines and alerts in ADF.
Implementing new enhancements for the LBU.
Analyzing and fixing issues raised by the business.
Updating JIRA and documenting in Confluence.
Ideal Candidate Profile
56 years of hands-on experience in Azure data engineering, with strong exposure to ADF, Databricks, and Synapse.
Solid programming skills in SQL, Python, and PySpark for building transformation logic and data engineering frameworks.
Experience working with SFTP-based data extraction and file-based integrations.
Familiarity with CI/CD practices using Azure DevOps.
Strong analytical and issue-resolution skills, with experience supporting business/production issues.
Good communication skills for documentation (Confluence) and task tracking (JIRA).
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.