- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build and optimize large-scale distributed data processing solutions using Apache Spark and PySpark.
- Develop and manage Lakeflow Declarative Pipelines for modern data engineering workflows.
- Implement efficient SQL-based data transformation and integration processes.
- Work with data lake and lakehouse architectures to support enterprise analytics and reporting needs.
- Optimize pipeline performance, scalability, reliability, and cost efficiency.
- Collaborate with cross-functional teams including architects, analysts, and business stakeholders.
- Implement CI/CD processes and version control practices using Git and related tools.
- Ensure data quality, governance, security,
and compliance best practices are followed.
- Troubleshoot and resolve issues in development, testing, and production environments.
- Participate in Agile/Scrum ceremonies and contribute to continuous improvement initiatives.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.