We are seeking a highly skilled Senior Data Engineer to architect, build, secure, and optimize end-to-end data pipelines and enterprise-grade data platforms. The ideal candidate brings deep expertise in Python, Snowflake, modern cloud ecosystems, data warehousing design, and DevSecOps-led automation.
You will mentor junior engineers, drive technical decision-making, and lead solutions that improve data quality, performance, and reliability across the organization.

Data Engineering & Pipelines
- Design, develop, and optimize scalable ELT/ETL pipelines using Python and SQL.
- Build real-time, near-real-time, and batch frameworks using cloud-native services.
- Implement incremental loads, CDC, SCD, schema evolution, and orchestration best practices.

Snowflake Engineering
- Architect and manage Snowflake environments: warehouses, databases, schemas, resource monitors, RBAC, zero-copy clones.
- Implement Snowflake Tasks, Streams, Pipes (Snowpipe) for event-driven data workflows.
- Optimize compute cost and query performance using clustering, micro-partitioning, caching, and warehouse sizing.





Cloud Engineering (AWS | Azure | GCP)
- Build and maintain solutions using cloud-native compute/storage components:
- Azure: Blob Storage, ADLS, Functions, ADF, Purview

Data Warehousing & Modeling
- Design enterprise-grade Data Warehouses, Data Marts, and Semantic Layers.
- Implement Kimball, Data Vault, and modern ELT-first design patterns.
- Work closely with BI/ML teams to operationalize features and analytics models.

DevSecOps & Platform Engineering
- Implement CI/CD pipelines for data engineering code (GitHub Actions / Azure DevOps / GitLab CI).
- Enforce DevSecOps practices:
- secret scanning
- IaC security gates
- dependency scanning
- policy-as-code (OPA/Conftest)
- Build infrastructure using Terraform / Azure Bicep / CloudFormation.

Automated Testing & Data Quality
- Implement data unit testing, schema validation, and contract enforcement:
- pytest
- Outstanding Expectations

📌 Senior Software Engineer (Chennai)
🏢 Nielseniq
📍 Chennai