15 Nov
|
HCLTech
|
Thiruvananthapuram
15 Nov
HCLTech
Thiruvananthapuram
## Job Summary We are looking for a Senior Data Engineer with strong Databricks, SQL and data warehousing experience to lead the migration of an existing Oracle-based data warehouse to Databricks on the Allianz Data Platform (ADP). Experience with contemporary data engineering practices and AI-enabling data platforms is highly desirable. ##
Key Responsibilities
Analyse the current Oracle data warehouse (including PostgreSQL source feeds and SAS ETL–generated structures) and design the target model in Databricks.
• Build and optimise ETL/ELT pipelines in Databricks (Spark SQL, PySpark) to migrate core data warehouse tables and datamarts from Oracle to Databricks.
• Translate Oracle SQL/PL-SQL logic into Databricks Spark SQL and PySpark, ensuring functional equivalence and performance optimisation.
• Ensure data quality, performance, and reliability of the new Databricks-based data warehouse, enabling downstream analytics and AI/ML use cases.
• Collaborate closely with the Enterprise Architect, ADP platform team and business stakeholders to align with architecture, standards, and best practices.
• Document migration patterns, data models, and operational procedures to support long-term maintainability and reusability. ## Skill Requirements Graduate or Postgraduate in Engineering or a Post graduate in non-engineering disciplines .
• Experience of 5-9 years of experience in ITIL process/application support/Deployment activities.
• Basic knowledge on database
• Strong hands-on experience with Databricks (Spark SQL, PySpark, Delta Lake, jobs, clusters).
• Proven background in data warehousing (dimensional modelling, ETL/ELT, performance tuning).
• Solid experience with Oracle DWH (SQL, PL/SQL) and migrations from Oracle or similar legacy DWH platforms.
• Experience integrating PostgreSQL as a data source into a central DWH/Lakehouse.
• Ability to reverse-engineer existing ETL logic (e.g. SAS jobs) and translate it into Databricks pipelines, even if not directly working on SAS BI reports.
• Strong communication and stakeholder management skills in an international, multi-vendor setup.
• Good understanding of how high-quality, well-modelled data underpins advanced analytics and AI/ML capabilities. ## Must Have Skill AWS Analytics,AWS Core Services,Amazon Relational Database Services,Data Lakes,Data Pipelines,ETL Tool
- Others ## Good to Have Skill ## Other Requirements Strong Analytical, Logical and Communication Skills (written, oral, one to one etc).
• Professional, Innovative, Drive for continuous improvement.
• Positive attitude, proactively provides solutions
📌 Track Lead (Thiruvananthapuram)
🏢 HCLTech
📍 Thiruvananthapuram