Pipeline Orchestration: Use Fabric Data Factory to orchestrate Azure Databricks jobs ( Python scripts).
Lakehouse Implementation: Design and maintain using Delta Lake.
Data Integration: Land data from Databricks directly into Fabric OneLake or create shortcuts to access Databricks Delta files without moving data.
ETL/ELT Development: Develop high-performance processing applications using PySpark, Spark SQL, and Python.
Key Technical Skills
Cloud Platforms: Deep expertise in the Azure Data Stack, specifically Azure Data Lake Storage (ADLS) and Azure Synapse.
Languages: Proficiency in Python, SQL, and occasionally Scala or Java for custom Spark development.
Fabric Tools: Hands-on experience with Fabric Lakehouse, Data Warehouse, and Power BI integration (Direct Lake mode).
DevOps: Familiarity with CI/CD pipelines (Azure DevOps/GitHub Actions) for automating data workload deployments.