19 Aug
|
Innova ESI
|
Maharashtra
19 Aug
Innova ESI
Maharashtra
Data Engineer/ Senior Data Engineer – Snowflake, Snowpark & DBT
Job Title: Senior Data Engineer
Experience: 5+ Year to 11 Year
Location: Pune
Job Overview
We are looking for an experienced Senior Data Engineer with strong expertise in Snowflake, Snowpark, DBT, Python, and SQL to design, develop, and optimize scalable data engineering solutions. The candidate will be responsible for building data ingestion pipelines, implementing ETL/ELT processes, developing data models, and ensuring data quality, governance, security, and performance.
The role requires strong hands-on experience with Snowflake ecosystem capabilities, including Snowpark, Snowpipe, Streams, Tasks, Time Travel, Cloning, SnowSQL, Snowsight, OpenFlow, stored procedures, UDFs, and data sharing.
Mandatory Skills
Candidates must have hands-on experience in:
- Snowflake
- Snowpark
- DBT (Core or Cloud)
- Python
- SQL
- ETL/ELT and Data Engineering concepts
- Data Warehousing concepts
- Data ingestion pipeline development
- CDC / SCD Type 2
- Airflow or other orchestration tools
- Cloud platform – AWS / Azure / GCP
- Snowflake Cortex AI / Cortex AI capabilities
Key Responsibilities
- Develop, implement, and optimize scalable data pipelines using Snowflake, with a focus on Cortex AI capabilities.
- Design and implement ETL/ELT pipelines for extracting, transforming, and loading data from multiple sources into Snowflake.
- Build reliable data ingestion pipelines while ensuring data integrity, quality, and accuracy.
- Utilize Snowflake capabilities including Snowpark, Streams, Tasks, Snowpipe, SnowSQL, bulk copy, Time Travel, Cloning, Optimizer, Metadata Manager, Data Sharing, Stored Procedures, and UDFs.
- Work with Snowsight, OpenFlow, and Snowpark for data processing and engineering solutions.
- Develop and maintain DBT models, tests, documentation, and data lineage.
- Implement CDC and SCD Type 2 solutions for incremental data processing and historical data management.
- Develop productive SQL queries and optimize Snowflake workloads for performance and cost.
- Build and manage workflows using Airflow or other orchestration tools.
- Apply software engineering best practices including Git-based version control, code reviews, automated testing, and CI/CD.
- Maintain data documentation, metadata, lineage, governance, and transformation standards.
- Ensure data security, privacy, compliance, and access controls.
- Translate business requirements and discussions into technical design and data engineering solutions.
- Collaborate with cross-functional teams and effectively liaise with offshore teams to ensure quality and on-time delivery.
- Troubleshoot data quality and pipeline issues and implement appropriate solutions.
Required Technical Skills
- Snowflake: Strong hands-on experience with Snowflake data engineering and utilities.
- Snowpark: Hands-on experience developing data processing solutions using Snowpark.
- DBT: Strong experience with DBT Core or DBT Cloud, including models, testing, documentation, and lineage.
- Python: Strong programming and scripting skills for data engineering.
- SQL: Advanced SQL and query optimization.
- ETL/ELT:
Strong understanding and practical experience with data integration and pipeline development.
- Data Warehousing: Strong understanding of dimensional modeling and data warehouse concepts.
- CDC/SCD: Experience implementing CDC and SCD Type 2.
- Orchestration: Airflow or equivalent workflow management tools.
- Cloud: Experience with AWS, Azure, or GCP.
- Version Control: GitHub, GitLab, Azure Repos, or equivalent.
Good to Have
- Experience with data integration tools such as SSIS, Matillion, Talend, or Informatica.
- Experience with Snowflake Cortex AI and AI-driven data extraction/use cases.
- Experience with Snowflake OpenFlow.
- Knowledge of metadata management and data governance frameworks.
- Experience with CI/CD for DBT and Snowflake deployments.
Qualifications
- B.E./B.Tech/Master’s degree in Computer Science, Information Technology, Computer Engineering, or an equivalent qualification.
- Relevant experience as a Senior Data Engineer with strong DBT + Snowflake expertise.
- Excellent analytical, problem-solving, communication, and collaboration skills.
Skill Matrix
Snowflake
Mandatory
Snowpark
Mandatory
DBT Core / DBT Cloud
Mandatory
Python
Mandatory
SQL
Mandatory
Cortex AI
Mandatory
ETL/ELT
Mandatory
Data Warehousing
Mandatory
Airflow / Orchestration
Mandatory
AWS / Azure / GCP
Mandatory
CDC / SCD Type 2
Mandatory
GitHub / GitLab / Azure Repos
Good to Have
SSIS / Matillion / Talend / Informatica
Good to Have
Location: Pune
Budget: ₹13 LPA – ₹18 LPA
Experience: 5+ Years
Important: Candidates without strong hands-on experience in Snowflake, Snowpark, DBT, Python, SQL, and Cortex AI should not be considered.
📌 Data Engineer/Senior Data Engineer – Snowflake, Snowpark & DBT| Immediate joiner| Pune (Maharashtra)
🏢 Innova ESI
📍 Maharashtra