04 Aug
|
Birlasoft
|
Noida
Cloudops Area(s) of responsibility : Cloud Data Platform Engineer / SRE (Azure | Databricks | Fabric | Unity Catalog) _________________ Key Responsibilities 1.
Cloud Platform
Operations • Manage and optimize Azure workloads—ADLS, VNets, Key Vault, ADF, Synapse, Fabric, Databricks.
- Configure and maintain Databricks clusters, jobs, DLT pipelines, Delta Lake storage, and Unity Catalog policies.
- Operationalize Fabric Lakehouses, Pipelines, Warehouses, and Semantic Models for production workloads.
- Ensure robust platform governance across environments (DEV–QA–UAT–PROD).
- Infrastructure as Code &
- CI/CD • Build and maintain Terraform/Bicep templates for environment provisioning and configuration.
- Develop end-to-end CI/CD pipelines for Databricks, Fabric, and Azure components (ADO/GitHub).
- Automate deployment of notebooks, workflows, access policies, networking components, and Fabric artifacts.
- Enforce version control, release governance, and quality gates.
- FinOps, Cost Management &
- Capacity Planning • Implement FinOps dashboards, alerts, budgets, and spend governance practices.
- Perform Databricks and Fabric cost optimization—cluster sizing, autoscaling, idle management, job tuning.
- Conduct capacity planning for compute, storage, Fabric engines, and Databricks workloads.
- Develop cost-saving recommendations and automated consumption monitoring. 4.
Environment
Management, Security &
- Governance • Provision and manage Azure data environments with consistent policies and naming standards.
- Configure RBAC, ACLs, Unity Catalog grants, service principals, network security, Managed Identities.
- Implement governance standards for data access, lineage, audit logging, compliance, and risk mitigation.
- Ensure secure connectivity using Private Endpoints, VNET integration, and enterprise IAM controls.
- Monitoring, Observability &
- Platform Reliability (SRE) • Implement monitoring and alerting using Azure Monitor, Log Analytics, Databricks Metrics, Fabric Admin APIs.
- Build runbooks, dashboards, and automated remediation workflows for platform reliability.
- Conduct performance tuning of data workloads, Fabric pipelines, Databricks jobs, and storage layers.
- Lead incident management, root cause analysis, and environment stabilization efforts. ________________________________________ Required Skills &
Experience • 6–12 years in cloud data engineering, SRE, or platform engineering roles.
- Strong hands-on expertise with: o Azure Data Services (ADLS, ADF, Synapse, Key Vault, VNets) o Azure Databricks (clusters, jobs, Delta Lake, DLT, Unity Catalog) o Microsoft Fabric (Lakehouse, Pipelines, Warehouse, Dataflows) o Unity Catalog governance (catalogs, schemas, access policies, lineage) • Robust scripting and automation experience: Python, PowerShell, Bash, SQL, PySpark.
- Experience with Terraform/Bicep for IaC.
- Solid knowledge of Azure DevOps or GitHub Actions CI/CD pipelines.
- Proven FinOps experience with cost governance and optimization across cloud workloads.
- Experience in SRE practices—SLIs, SLOs, operational readiness, automated recovery. ________________________________________ Preferred Qualifications • Certifications in Azure Data Engineer, Azure DevOps Engineer, Databricks Data Engineer, FinOps Practitioner.
- Experience in highly regulated environments (BFSI, Healthcare, Retail).
- Understanding of zero-trust security models.
Experience Level Senior Level
📌 Senior Azure Data Engineer (Noida)
🏢 Birlasoft
📍 Noida