- Design, develop, and maintain data pipelines using Azure Data Factory (ADF).
- Build scalable data processing solutions using Azure Databricks, Python, and PySpark.
- Develop ETL/ELT workflows for data ingestion, transformation, and loading.
- Work with Azure Data Lake Storage (ADLS), Azure SQL, and other Azure data services.
- Optimize Spark jobs and Databricks notebooks for performance and scalability.
- Implement data quality checks, validation rules, and monitoring processes.
- Create and manage end-to-end data integration solutions across multiple data sources.
- Collaborate with business stakeholders and data teams to understand data requirements.
- Troubleshoot and resolve data pipeline and performance issues.
- Follow coding standards, version control, and CI/CD best practices.
- Support data warehousing and analytics initiatives.
- Prepare technical documentation and maintain deployment artifacts.