15 Aug
|
Persistent Systems
|
Pune City
15 Aug
Persistent Systems
Pune City
About Position:
We are looking for an experienced AWS Databricks professional with 10+ years of experience in Data Engineering, Platform Engineering, and Cloud Architecture. The role involves designing and implementing scalable Lakehouse solutions on Databricks and AWS, building robust data platforms, driving AI-enabled data initiatives, and enabling enterprise-wide data governance, security, and automation. The ideal candidate should have solid expertise in AWS, Databricks, SQL, Performance Tuning, and PySpark, along with experience in Banking and Financial Services domains.
- Role: AWS Databricks Engineer
- Location: All Persistent Locations
- Experience: 10+ years
- Job Type: Full-Time Employment
What You'll Do:
- Design and implement scalable Lakehouse architecture using Databricks, Delta Lake / Iceberg, and Cloud Storage.
- Integrate Databricks with Cloud Services (AWS/Azure) including storage, IAM, and event services.
- Implement Security, Data Lineage, Access Governance, and Auditing Standards in Unity Catalog
- Drive implementation of AI for Data
- Develop & implement reusable pipelines and frameworks for the entire lifecycle of data
- Develop Automations for BAU Tasks
- Enable various Platform features and integrations with other platforms
- Assist service lines to design a solution architecture for specific use cases
- Design reusable design patterns for the platform
- Define and enforce data quality, observability, and FinOps monitoring.
- Adoption Engineering, contribute towards setting up a migration factory for legacy platform to Databricks
- Develop robust ETL/ELT pipelines using PySpark, Spark SQL, and Databricks Notebooks.
Expertise You'll Bring:
- 10 + years of IT experience with min 2 to 4 yrs exp in designing, building, and optimizing large-scale data processing solutions on the Databricks Platform
- Databricks, AWS Cloud Platform, Platform Engineering, Data Engineering, Architecture & Design, Terraform/Ansible, Mosaic AI & MLOps, CI/CD, GitH
📌 AWS Databricks Engineer (Pune City)
🏢 Persistent Systems
📍 Pune City