12 Oct
|
Mphasis
|
Bengaluru
Role & responsibilities An experienced Data Warehouse Data Engineer with solid expertise in Databricks Lakehouse , PYSPARK application development to join the Mphasis Data Platform team.
The role focuses on designing, developing, and optimizing data pipelines, data models, and governance frameworks for Residential (Resi) & Mortgage Servicing Rights (MSR) data warehouse initiatives. The ideal candidate combines strong Databricks engineering skills with positive to have on mortgage data domain knowledge, and can collaborate with cross-functional teams to deliver high-quality, governed, and scalable solutions. Note that the Databricks flavor used in Azure Databricks
Key Responsibilities
- Data Ingestion & Processing
- Develop scalable, metadata-driven ingestion pipelines using PySpark ,Python, SQL in Databricks.
- Automate batch and streaming ingestion from servicer files and internal systems.
- Handle structured/unstructured data (loan tapes, CSV, JSON, parquet, Delta Lake).
- Optimization & Performance
- Optimize pipelines for performance and cost (DBU management, caching, cluster configs, partition strategies).
- Implement job monitoring,
logging, and ing for proactive issue resolution.
- Collaboration & Delivery
- Work with Data Architects, Governance Officers, and Business Analysts to align technical delivery with business rules.
- Follow CI/CD practices (Dev QA Prod migration) using Databricks Repos, GitHub, and Azure DevOps.
- Participate in code reviews, design walkthroughs, and audits to ensure compliance with standards.
- Technical skills
- 8+ years of experience in Data Engineering / DWH development.
- Hands-on experience with Databricks (PySpark, Python,Delta Lake, Unity Catalog).
- Strong SQL expertise (T-SQL and SparkSQL) for data transformations and analytics the Data Engineer might need to reverse engineer quite a few SQL SPs.
- Exposure to Delta Live Tables (DLT), streaming pipelines, and structured streaming.
- Experience with Azure Data Lake (OneLake/ADLS) as storage layer.
- Knowledge of Azure DevOps/GitHub for version control & CI/CD.
Preferred candidate profile
Perks and benefits
📌 Azure Data Engineer (Bengaluru)
🏢 Mphasis
📍 Bengaluru