15 Aug
|
Datametica
|
Pune
Matallion Data Lead
Experience : 7+ Years
Location: Pune
We are seeking a Senior Data Engineer/ Lead to lead a strategic migration of our enterprise data platform from Matillion ETL to a native Google Cloud Platform (GCP) stack. The ideal candidate is a Matillion expert who understands the "under the hood" logic of graphical ETL jobs and can translate that logic into scalable, maintainable, and cost-effective code. You will be the bridge between our legacy systems and our future state, utilizing AI-assisted migration tools to accelerate the conversion of complex transformations into native BigQuery, Dataform, or Python-based workflows.
Key Responsibilities
Reverse Engineering: Deeply analyze existing Matillion Orchestration and Transformation jobs to understand business logic, dependencies, and complex iterations.
Migration Strategy: Lead the "Extract & Translate" phase, determining whether a job should be refactored into BigQuery SQL (Dataform), Cloud Dataflow (Python), or Cloud Composer (Airflow).
AI-Enabled Conversion: Leverage LLMs and AI coding assistants to parse Matillion metadata (JSON/XML) and automate the generation of native SQL and Python code.
Mentorship & Documentation: Act as the subject matter expert (SME)
to explain Matillion logic to the broader GCP engineering team and document legacy rules for the new architecture.
Validation: Design and execute data reconciliation frameworks to ensure parity between Matillion outputs and current GCP native outputs.
Required Experience & Skills
Matillion Mastery: 3+ years of advanced experience with Matillion ETL (specifically Grid Variables, Iterators, and Python script components).
Data Architecture: Solid understanding of Data Warehousing principles (Star/Snowflake Schema) and ELT vs. ETL methodologies.
SQL Expertise: Expert-level SQL skills, including the ability to refactor complex GUI-based logic into high-performance stored procedures or views.
Python Proficiency: Solid Python background for handling complex data logic and building orchestration scripts.
AI Prompting/Automation: Experience (or high interest) in using Generative AI tools to automate code conversion and documentation tasks.
Preferred (Nice-to-Have) Skills
GCP Experience: Familiarity with BigQuery, Dataform, or Airflow (we are willing to train the right Matillion expert on the GCP ecosystem).
Metadata Engineering: Experience parsing XML/JSON job definitions to create automated documentation or migration scripts.
📌 Matillion Lead Pune
🏢 Datametica
📍 Pune