Job Summary

We are looking for an experienced Azure Data, Databricks and AI Platform Engineer to manage and support enterprise data and AI platforms hosted on Microsoft Azure. The role requires hands-on experience across Azure Databricks, Azure Data Factory, Azure Machine Learning, Azure AI Foundry, Azure OpenAI, Azure SQL, ADLS Gen2, Azure Storage, Azure DevOps, infrastructure as code using Bicep and Terraform, identity and access management, monitoring, security, and cloud connectivity. The successful candidate will be responsible for maintaining platform availability, security, performance, compliance, and operational stability. The engineer will also support workplace provisioning, CI/CD pipelines, data product onboarding, platform upgrades, incident resolution, disaster recovery, and lifecycle management across the Azure data and AI ecosystem.

Key Responsibilities

- Administer and support enterprise Azure data and AI platform environments.
- Provision, configure, maintain, and decommission Azure resources through automated infrastructure-as-code deployments using Bicep and Terraform, based on approved project and platform requirements.
- Administer Azure Databricks workspaces, clusters, compute resources, runtime versions, repositories, Unity Catalog, jobs, and workflows.
- Monitor Databricks environments and troubleshoot cluster, runtime, job, workflow, access, dependency, authentication, and connectivity issues.
- Administer Azure Data Factory environments, pipelines, Integration Runtime infrastructure, and associated platform configurations.
- Monitor Integration Runtime nodes and manage upgrades, restarts, availability issues, and recovery activities.
- Support Azure Machine Learning, Azure AI Foundry, Azure OpenAI, Databricks Model Serving, Databricks Apps, Databricks Lakebase, and Databricks MCP platform environments.
- Manage platform configurations, access, identities, secrets, connectivity, monitoring, security, and compliance requirements for supported data and AI services.
- Administer Azure SQL Database, SQL Server, and Azure SQL dedicated pool resources, including access, connectivity, performance, and availability support.
- Manage Azure Storage Accounts, ADLS Gen2 containers, permissions, connectivity, and resource lifecycle activities.
- Develop, maintain, validate, and troubleshoot reusable Bicep modules and Terraform configurations for consistent provisioning of Azure data, Databricks, AI, storage, networking, monitoring, identity, and security resources across environments.
- Integrate infrastructure-as-code deployments with Azure DevOps CI/CD pipelines, including parameter management, state management,



deployment approvals, validation, rollback planning, and post-provisioning checks.
- Support CI/CD pipelines across Azure DevOps, Azure Databricks, and Azure Data Factory.
- Manage repositories, service connections, deployment workflows, release pipelines, environment configurations, and production approval gates.
- Troubleshoot deployment failures involving authentication, authorization, repository access, service connections, environment dependencies, and platform availability.
- Perform approved deployment reruns, pipeline recovery, release coordination, and post-deployment validation.
- Manage access provisioning using Microsoft Entra ID users, security groups, service principals, managed identities, Azure RBAC, OAuth, tokens, and secrets.
- Administer Azure Key Vault integrations and support secret rotation, credential maintenance, and secure application authentication.
- Support Unity Catalog access for catalogs, schemas, tables, views, groups, and repositories.
- Monitor platform availability, resource utilization, cluster health, job execution, pipeline status, storage capacity, token usage, and cloud costs.
- Use Azure Monitor, Log Analytics, Application Insights, and enterprise monitoring tools to investigate alerts and platform issues.
- Respond to production incidents and cloud platform outages, perform technical diagnosis, execute recovery actions, and validate service restoration.
- Troubleshoot connectivity issues involving VNets, private endpoints, service endpoints, DNS, routing, firewalls, network security controls, wallets, DSNs, and platform integrations.
- Support vulnerability remediation, security reviews, platform patching, access reviews, database activity audits, and compliance-control activities.
- Prepare technical evidence and documentation for internal and external audits.
- Participate in disaster recovery testing, Databricks runtime upgrades, platform maintenance, and service-restoration exercises.
- Support the onboarding and decommissioning of data products across Azure and Databricks environments.
- Coordinate workspaces, storage, access, identities, secrets, metadata, pipelines, monitor.

Skill Requirements

- Strong hands-on experience in Microsoft Azure platform administration.




- Experience administering Azure Databricks workspaces, clusters, runtimes, jobs, workflows, repositories, and Unity Catalog.
- Strong experience with Azure Data Factory, pipelines, linked platform dependencies, and Integration Runtime.
- Experience supporting Azure Machine Learning, Azure AI Foundry, Azure OpenAI, or similar enterprise AI platform services.
- Strong hands-on experience with infrastructure as code using Bicep and Terraform, including reusable modules, environment-specific parameterization, remote state management, deployment validation, and automated Azure resource provisioning.
- Experience with Azure DevOps repositories, service connections, CI/CD pipelines, release pipelines, and deployment troubleshooting.
- Experience administering Azure SQL Database, SQL Server, or Azure SQL dedicated pools.
- Experience managing Azure Storage Accounts and ADLS Gen2.
- Strong understanding of Microsoft Entra ID, Azure RBAC, security groups, service principals, managed identities, OAuth, tokens, and application authentication.
- Experience with Azure Key Vault, secrets management, credential rotation, and secure platform integrations.
- Hands-on experience with Azure Monitor, Log Analytics, Application Insights, diagnostic settings, alerts, and operational dashboards.
- Good understanding of Azure networking, including VNets, subnets, private endpoints, service endpoints, DNS, routing, firewalls, and network security controls.
- Experience supporting security, vulnerability remediation, access governance, audit compliance, disaster recovery, and production change management.
- Experience with ServiceNow or an equivalent IT service management platform.
- Working knowledge of SQL, Python, PySpark, Databricks notebooks, and automation technologies.
- Strong experience troubleshooting business-critical cloud, data, and AI platform environments.

Other Requirements

- Maintain data product inventories, ownership mappings, operational dashboards, and CMDB records.
- Perform Azure resource consumption and cost analysis to support platform usage visibility.
- Administer Azure Automation workers and agents, including monitoring, upgrades, restarts, maintenance, and recovery.
- Manage incidents and service requests using ServiceNow or equivalent IT service management tools.
- Work with cloud infrastructure, data engineering, application, networking, identity, cybersecurity, Microsoft, and vendor teams to resolve complex platform issues.
- Maintain platform documentation, standard operating procedures, troubleshooting guides, and knowledge-base articles.

📌 SeniorAdministrator - Azure DevOps, Terraform (India)
🏢 HCL Technologies
📍 India