PySpark, Azure Data Factory Responsibilities Design, develop, and maintain data pipelines using
Azure Data Factory . Manage and optimize data storage using
Azure Data Lake Gen 2 . Build and process large-scale data solutions with
Azure Databricks
and
Apache Spark . Create interactive reports and dashboards in
Power BI
for business insights. Collaborate with cross-functional IT and business teams to translate requirements into data solutions. Ensure data quality, security, and compliance in line with organizational standards. Core Skills Required Azure Data Factory : Experience in building and managing data pipelines. Azure Data Lake Gen 2 : Proficient in data storage and management within Azure's data lake workplace. Azure Databricks / Apache Spark : Hands-on skill with distributed data processing, transformations, and analytics. Power BI : Expertise in data visualization and reporting.