Educational Requirements

Bachelor of Engineering

Service Line

Data Analytics Unit

Responsibilities
Administer and manage Azure/AWS Databricks workspaces.
Configure and maintain clusters, including creation, scaling, and optimization.
Manage user access, roles, and permissions using RBAC and integration with identity providers (Azure AD, etc.).
Monitor cluster performance and optimize cost, resource utilization, and job execution.
Implement and manage job scheduling, workflows, and pipelines.
Ensure data security and governance, including data encryption and access policies.
Integrate Databricks with data sources such as ADLS, S3, Snowflake, or SQL databases.
Troubleshoot issues related to jobs, clusters, notebooks, and integrations.
Support CI/CD pipelines and DevOps processes for Databricks deployments.
Monitor usage through logs, metrics, and audit trails.
Maintain documentation for architecture, configurations, and processes.
Collaborate with data engineers, data scientists, and DevOps teams.





Additional Responsibilities: Technical and Qualified Requirements:
25 years of experience in Databricks Administration / Big Data Platforms.
Solid hands-on experience with Databricks (Workspace, Clusters, Jobs, Notebooks).
Experience with Apache Spark (PySpark/Scala) and distributed data processing.
Knowledge of cloud platforms (Azure preferred, AWS/GCP also acceptable).
Experience with data storage systems like ADLS, S3, Blob Storage.
Positive understanding of data engineering concepts, ETL/ELT pipelines.
Familiarity with security best practices (IAM roles, data encryption, private endpoints).
Proficiency in SQL and scripting languages (Python, Shell, etc.).
Experience in performance tuning and cost optimization for clusters and workloads.

Preferred Skills:
Technology->Data Engineering->Databricks

📌 Databricks Admin Karnataka
🏢 Infosys
📍 Karnataka