Azure Databricks, PySpark, Azure Data Factory
Responsibilities
Design, develop, and maintain data pipelines using Azure Data Factory.
Manage and optimize data storage using Azure Data Lake Gen 2.
Build and process large-scale data solutions with Azure Databricks and Apache Spark.
Create interactive reports and dashboards in Power BI for business insights.
Collaborate with cross-functional IT and business teams to translate requirements into data solutions.
Ensure data quality, security, and compliance in line with organizational standards.
Core Skills Required
Azure Data Factory: Experience in building and managing data pipelines.
Azure Data Lake Gen 2: Proficient in data storage and management within Azure's data lake setting.
Azure Databricks / Apache Spark: Hands-on skill with distributed data processing, transformations, and analytics.
Power BI: Expertise in data visualization and reporting.