We are seeking a Lead Data Engineer with 7-10 years of experience to join our Data Platform team. This role will report to the Manager of data engineering and be involved in the planning, design, and implementation of our centralized data warehouse solution for ETL, reporting and analytics across all applications within the company.
Qualifications:
- Deep knowledge and experience working with Python/Scala and PySpark
- Experienced inAzure Data bricks, Azure Blob Storage, Azure Data Lake, Delta lake.
- Experience working with Spark SQL
- Experience in creating pipelines and databricks dashboards
- Experience with Azure cloud environments
- Experience with acquiring and preparing data from primary and secondary disparate data sources
- Experience working on large scale data product implementation, responsible for technical delivery.
- Good understanding of OOPs and design patterns
- Experience working with agile methodology preferred.
- Healthcare industry experience preferred.
Responsibilities:
- Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions
- Work with other team with deep experience in ETL process, distributed microservices, and data science domains to understand how to centralize their data
- Share your passion for staying experimenting with and learning recent technologies.
- Perform thorough data analysis, uncover opportunities, and address business problems.