Job Description

Our Technology teams are working on our new validation platform. Our clients around the world rely on data and insights to innovate and grow.

As a Senior Engineer, you’ll be part of a team of smart, highly skilled technologists who are passionate about learning and supporting cutting-edge technologies such as Snowflake Tasks, Streams, Pipes (Snowpipe), External Tables, components: Azure: Blob Storage, ADLS, Functions, ADF, Purview

Roles and Responsibilities

Data Engineering & Pipelines

- Design, develop, and optimize scalable ELT/ETL pipelines using Python and SQL.

- Build real-time, near-real-time, and batch frameworks using cloud-native services.

- Implement incremental loads, CDC, SCD, schema evolution, and orchestration best practices.

Snowflake Engineering

- Architect and manage Snowflake environments: warehouses, databases, schemas, resource monitors, RBAC, zero-copy clones, clusters, micro partition.

- Implement Snowflake Tasks, Streams, Pipes (Snowpipe), External Tables for event-driven data workflows.

- Optimize compute cost and query performance using clustering, micro-partitioning, caching, and warehouse sizing.

Cloud Engineering - Azure

- Build and maintain solutions using cloud-native compute/storage components:

- Azure: Blob Storage, ADLS, Functions, ADF, Purview





Data Warehousing & Modeling

- Design enterprise-grade Data Warehouses, Data Marts, and Semantic Layers.

- Implement Kimball, Data Vault, and contemporary ELT-first design patterns.

- Work closely with BI/ML teams to operationalize features and analytics models.

AI engineering

- Design, develop, and deploy AI and Generative AI solutions aligned with business requirements.

DevSecOps & Platform Engineering

- Implement CI/CD pipelines for data engineering code (GitHub Actions / Azure DevOps / GitLab CI).

- Enforce DevSecOps practices:

- secret scanning

- IaC security gates

- dependency scanning

- policy-as-code (OPA/Conftest)

- Build infrastructure using Terraform / Azure Bicep / CloudFormation.

Automated Testing & Data Quality

- Implement data unit testing, schema validation, and contract enforcement:

- pytest

- Great Expectations/dbt tests

- automated data profiling

- Build automated quality dashboards, lineage, and SLA monitoring.

Leadership & Collaboration

- Lead design reviews, code reviews, and data platform roadmap discussions.

- Mentor junior engineers and enforce engineering excellence.

- Partner with Product, Data Science, and Business teams to deliver data-driven solutions.

📌 Senior Software Engineer - Python with Snowflake (Chennai)
🏢 Nielseniq
📍 Chennai