Roles and Responsibilities
Design, develop, and maintain large-scale data pipelines using Azure Databricks, Pyspark, and SQL.
Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time.
Develop complex data models and ETL processes to extract insights from various sources.
Troubleshoot issues related to data quality, performance, and security in the pipeline.