Job Summary: We are seeking a skilled Databricks Platform Administrator with a strong background in Azure Databricks to manage and optimize our data analytics platform. The ideal candidate will have extensive experience in administering Databricks environments, ensuring high availability, performance, and security. You will collaborate with data engineers, data scientists, and other stakeholders to support data-driven decision-making across the organization.
Responsibilities
Administer and manage Azure Databricks environments, ensuring optimal performance and reliability.
Monitor and troubleshoot Databricks clusters, jobs, and notebooks to ensure smooth operations.
Implement security best practices, including user access management and data governance.
Collaborate with data engineering and data science teams to support their analytics needs.
Develop and maintain documentation for Databricks configurations, processes, and best practices.
Stay updated with the latest Databricks features and Azure services to enhance platform capabilities.
Provide training and support to team members on Databricks usage and best practices.
Participate in capacity planning and performance tuning of the Databricks setting.
Mandatory Skills
Proven experience with Azure Databricks administration.
Strong understanding of data engineering concepts and practices.
Experience with Spark, SQL, and data processing frameworks.
Familiarity with Azure cloud services and architecture.
Knowledge of security and compliance standards in cloud environments.
Preferred Skills
Experience with data visualization tools such as Power BI or Tableau.
Familiarity with CI/CD practices and tools for data pipelines.
Knowledge of Python, Scala, or R for data processing and analysis.
Experience with monitoring and logging tools for cloud environments.