Responsibilities

- Design and optimize scalable data engineering solutions on Azure Databricks that enable advanced analytics and AI workloads for enterprise stakeholders and downstream applications
- Configure and administer Databricks Unity Catalog to establish centralized governance robust access controls and consistent data security policies across all workspaces and environments
- Develop automate and monitor Databricks Workflows that orchestrate complex data pipelines ensuring reliable scheduling error handling and recovery for critical business processes
- Implement and manage Databricks Delta Lake structures to support incremental data ingestion versioning and time travel capabilities that improve data quality and analytical flexibility
- Build fine tuned Databricks SQL queries dashboards and data models that deliver actionable insights to business teams while maintaining effective performance on large datasets
- Collaborate with data scientists analysts and application teams to translate analytical requirements into well structured data solutions that accelerate experimentation and deployment
- Document data models pipeline logic governance rules and operational procedures in a clear concise manner to promote transparency maintainability and knowledge sharing across teams
- Apply rigorous data quality checks validation frameworks and monitoring practices to ensure that curated datasets remain accurate consistent and fit for AI and reporting use cases
- Partner with cloud infrastructure and security teams to align Databricks configurations networking and identity controls with enterprise standards and regulatory expectations
- Optimize cluster configuration storage layout and resource utilization in Azure Databricks to balance performance reliability and cost efficiency for production and development workloads




- Contribute to continuous improvement initiatives by evaluating new Databricks features and best practices then guiding adoption that strengthens data platform capability and resilience
- Support domain oriented data solutions that can incorporate life and annuities insurance concepts where relevant helping business teams better understand risk policy behavior and customer outcomes
- Champion responsible and ethical use of data by promoting governance practices that protect customer information support regulatory compliance and enhance societal trust in data driven decisions

Qualifications

- Demonstrate strong hands on expertise in Azure Databricks including consistent practical experience with Databricks Unity Catalog Workflows SQL and Delta Lake in complex environments
- Bring solid experience in designing and operating large scale data pipelines with a minimum of nine years in data engineering or AI data specialist roles and exposure to modern cloud data stacks
- Show familiarity with life and annuities insurance data structures processes and metrics as a beneficial domain skill that can deepen the impact of analytics solutions for insurance stakeholders
- Exhibit advanced proficiency in structured query development performance tuning and data modeling practices that support reporting machine learning and self service analytics
- Apply sound knowledge of data governance security and compliance concepts to configure Unity Catalog permissions data classification and audit readiness for enterprise usage
- Demonstrate strong problem solving and debugging skills to investigate pipeline failures data anomalies and performance bottlenecks while driving sustainable corrective actions
- Communicate clearly with technical and nontechnical partners providing explanations of data solutions trade offs and risks that support informed decision making and collaboration
- Manage work effectively in an office based day shift model by coordinating with cross functional teams aligning priorities and ensuring consistent availability for stakeholder interactions

📌 Azure Databricks (Chennai)
🏢 Cognizant Technology Solutions India
📍 Chennai