Role & responsibilities
Build and maintain data pipelines and ETL workflows using Azure Data Factory and Databricks.
Support data integration from multiple sources into Azure Data Lake and Synapse settings.
Work with senior engineers to develop and optimize data transformation scripts in Python or PySpark.
Participate in data migration and modernization projects to Azure Cloud.
Ensure proper data validation, error handling, and logging within pipelines.
Implement basic security configurations using Key Vault and role-based access control.
Document pipeline design, configurations, and operational procedures.
Support DevOps-based deployment, CI/CD integration, and version control.
Troubleshoot and resolve data flow or pipeline issues during development and testing.
Collaborate with business analysts, architects, and QA teams to ensure end-to-end data delivery.
Continuously learn current Azure features and tools to improve solution efficiency.