- Experience in Pyspark and Azure Databricks.
- Proficiency in integrating, transforming, and consolidating data from various structured and unstructured data sources.
- Positive experience in SQL or native SQL query languages.
- Strong experience in implementing Databricks notebooks using Python.
- Good experience in Azure Data Factory, ADLS, Storage Services, Serverless architecture, Azure functions.
- Advanced working knowledge in building robust data pipelines using ADF
- Experience in building processes supporting data transformation, data structures, metadata, dependency and workload management