18 Sep
|
Insight Global
|
Bengaluru
18 Sep
Insight Global
Bengaluru
Day to day:
Insight Global is looking for an experienced Snowflake Data Engineer to design, build, and optimise modern data pipelines and cloud-based analytical environments. The ideal candidate will have strong expertise in SQL, Python, Snowflake features (Warehouses, Streams, Tasks, Time Travel), and cloud platforms such as Azure. This role will support enterprise data initiatives, ensure high performance, and enable advanced analytics including AI/ML integrations.
Key Responsibilities
Snowflake Development & Engineering
- Develop high‑performance SQL including advanced ANSI SQL, stored procedures, and user-defined functions.
- Build and manage Snowflake virtual warehouses, ensuring correct sizing, scaling, and multi‑cluster configurations.
- Optimise Snowflake workloads using clustering keys, caching strategies, warehouse configuration, and query tuning.
- Implement Streams and Tasks for real‑time ingestion, change data capture (CDC), and automated scheduling.
- Leverage Time Travel and Fail‑Safe features for data recovery, auditing, and historical analysis.
Data Pipelines & Integration
- Build and maintain ETL/ELT pipelines, including Snowflake ingestion using Snowpipe.
- Use Python for scripting, workflow automation, and in‑database processing through Snowpark.
- Integrate Snowflake with cloud storage, networking, and compute resources across Azure and AWS.
Data Architecture & Warehousing
- Apply strong data warehousing principles and dimensional modelling techniques.
- Ensure data models support analytics, reporting, and scalable business intelligence use cases.
Security, Governance & Compliance
- Implement data governance practices including RBAC, data access controls,
encryption, and secure data sharing.
- Configure secure data exchange using Snowflake Secure Data Sharing.
AI/ML & Emerging Technologies
- Integrate Snowflake with AI/ML solutions, including Snowflake Cortex and large language model (LLM) workflows.
- Use generative AI tools to automate data engineering and accelerate development tasks.
DevOps/DataOps
- Build and maintain CI/CD pipelines supporting data engineering workflows.
- Automate deployment, testing, and environment management using modern DevOps/DataOps practices.
Must Haves: 8 + years of experience in data engineering
Prove experience creating pipelines using Azure Data Factory
Strong hands-on experience in ETL/ELT orchestration and production operations
Hands on Snowflake experience
Proficiency with SQL: Advanced ANSI SQL, stored procedures, functions
Proficiency with Python: For scripting, automation, and using Snowpark for in-database processing
Experience building data pipelines using tools like Snowpipe
Data modelling experience
Expertise with Azure, AWS (storage, networking)
Strong DevOps/DataOps experience (CI/CD, automation)
Experience with OLTP databases such as PostgresDB or MS SQL Server or Azure SQL
Pluses:
Performance Tuning: Caching, clustering keys, virtual warehouses
Data Governance: Role-Based Access Control (RBAC), data sharing, encryption
Virtual Warehouses: Sizing, scaling, multi-cluster warehouses
Streams & Tasks: Change data capture and scheduling
Time Travel & Fail-Secure: Data recovery and historical access
Data Sharing: Secure data exchange
AI/ML Integration: Using Snowflake Cortex, integrating LLMs
Generative AI Tools: Leveraging AI for data tasks
📌 Senior Data Engineer (Bengaluru)
🏢 Insight Global
📍 Bengaluru