- Pipeline Development: Build and maintain scalable, secure ETL/ELT pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
- Data Processing: Utilize Apache Spark, PySpark, or Scala within Databricks to process large-scale structured and unstructured datasets.
- Data Lakehouse Management: Implement Medallion Architecture (Bronze, Silver, Gold layers) using Delta Lake on Azure Data Lake Storage (ADLS) Gen2.
- Data Warehousing: Design, model, and maintain high-performance data models and SQL pools in Azure Synapse.
Required Skills & Qualifications
- Cloud Platforms: Expert knowledge of Microsoft Azure (ADLS Gen2, Data Factory, Synapse, Databricks).
- Big Data Technologies: Proficiency in Databricks Spark, Delta Lake, and Synapse SQL.
- Programming Languages: Robust proficiency in SQL, Python, or Scala.
- Data Modeling: Expertise in dimensional modeling (star/snowflake schema) and data warehousing concepts.
- Certifications: Preferred certification in Microsoft Certified: Azure Data Engineer Associate (DP-203)