02 Aug
|
VMC Soft Technologies
|
Kolkata Metropolitan Area
02 Aug
VMC Soft Technologies
Kolkata Metropolitan Area
Databricks Administrator / Databricks Platform Engineer Immeediate joinner
Job Title: Databricks Administrator / Databricks Platform Engineer
Experience: 6 8 Years
Location: Bangalore (Preferred)
Job Summary We are seeking a skilled Databricks Administrator / Databricks Platform Engineer with 6 8 years of experience to manage and administer enterprise Databricks environments.
The ideal candidate will have strong expertise in Databricks Administration, Unity Catalog, PySpark, Data Governance, Security, CI/CD, and Data Migration . This role involves administering Databricks workspaces, implementing data governance and security policies, managing metadata, supporting deployments, and collaborating with stakeholders to ensure a secure, scalable, and high-performing data platform.
Key Responsibilities
- Administer and manage Databricks workspaces, clusters, jobs, notebooks, and platform configurations.
- Manage Unity Catalog, including creation, modification, and deletion of catalogs, schemas, tables, views, volumes, and functions.
- Configure and maintain metastore, object ownership, default properties, and workspace assignments.
- Implement and manage role-based access control (RBAC) using GRANT/REVOKE permissions.
- Enforce enterprise data governance policies using Unity Catalog.
- Implement fine-grained data security including:
- Row-Level Security (Row Filters)
- Column-Level Security (Column Masking)
- Attribute-Based Access Control (ABAC)
- Tags and Classifications
- Manage metadata including
- Data lineage
- Data cataloging
- Business metadata
- Data classification
- Compliance standards
- Troubleshoot access, permission,
workspace, and metadata issues.
- Perform ownership transfers and enforce platform governance standards.
- Design and execute secure cross-environment data migration (Production to Non-Production) using automated pipelines.
- Implement data masking and anonymization for sensitive information (PII, PHI, PCI) before migration.
- Build and optimize PySpark jobs for data processing and transformation.
- Upload and ingest data into Databricks from multiple source systems.
- Migrate data from relational databases and other enterprise platforms into Databricks.
- Build and support CI/CD pipelines for Databricks notebooks, jobs, and configuration deployments.
- Collaborate with Data Engineers, Data Architects, Security teams, and Business stakeholders.
- Monitor platform health, optimize performance, and resolve production issues.
- Document governance standards, operational procedures, and best practices.
Required Skills Mandatory Skills
- 6 8 years of IT experience.
- Strong hands-on experience in Databricks Administration.
- Expertise in Unity Catalog Administration.
- Strong experience with PySpark.
- Experience with Databricks Workspaces, Clusters, Jobs, and Notebooks.
- Strong understanding of Data Governance and Metadata Management.
- Experience implementing
- Role-Based Access Control (RBAC)
- Data Permissions
- Access Management
- GRANT/REVOKE
- Experience implementing
- Row-Level Security
- Column Masking
- Attribute-Based Access Control (ABAC)
- Strong understanding of
- Data Lineage
- Metadata Management
- Data Classification
- Data Catalog
- Experience migrating data across environments.
- Experience masking sensitive data (PII/PHI/PCI).
- Experience building CI/CD pipelines for Databricks deployments.
- Experience migrating data from relational databases into Databricks.
- Strong troubleshooting and production support experience.
- Excellent communication and stakeholder management skills.
Preferred Skills
- Azure Databricks or Databricks on AWS/GCP.
- Azure Data Factory (ADF), Apache Spark, Delta Lake.
- Git, Azure DevOps, GitHub Actions, Jenkins.
- SQL and performance tuning.
- Infrastructure as Code (Terraform/Bicep).
- Monitoring and logging tools.
- Agile/Scrum methodology experience.
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- Databricks certifications are highly preferred.
- Cloud certifications (Azure/AWS/GCP) are an added advantage.
Soft Skills
- Solid analytical and problem-solving abilities.
- Excellent communication and stakeholder engagement skills.
- Ability to work independently and in cross-functional teams.
- Strong documentation and knowledge-sharing skills.
- Customer-focused mindset with attention to quality and governance.
📌 Databricks Administrator / Databricks Platform Engineer (Kolkata Metropolitan Area)
🏢 VMC Soft Technologies
📍 Kolkata Metropolitan Area