Job Description

We are seeking a highly skilled **Principal Software Engineer – Data Engineering** to design, develop, and optimize scalable data platforms and pipelines. The ideal candidate will have robust expertise in **Python, SQL, Snowflake (or equivalent cloud data warehouse), cloud technologies, CI/CD practices, and AI-enabled data solutions**. This role requires technical leadership, architecture ownership, mentoring capabilities, and hands-on engineering excellence.

Key Responsibilities

Data Engineering & Platform Development

- Design and implement scalable, reliable, and secure data pipelines.
- Develop ETL/ELT solutions for processing large-scale structured and unstructured datasets.
- Build data ingestion frameworks supporting batch and near real-time processing.
- Design and optimize data models for analytics, reporting, and machine learning workloads.
- Ensure high data quality, governance, observability, and reliability across data platforms.

Cloud & Data Warehouse

- Architect and maintain cloud-native data solutions on AWS or Azure or GCP.
- Develop and optimize solutions using **Snowflake** or equivalent cloud data warehouse technologies.
- Implement data lake and lakehouse architectures.
- Optimize storage costs, compute utilization, and query performance.





Software Engineering Excellence

- Develop production-grade applications and frameworks using Python.
- Perform code reviews and establish engineering best practices.
- Drive architectural decisions and technology selection.
- Ensure adherence to security, scalability, and reliability standards.

CI/CD & DevOps

- Design and implement CI/CD pipelines for data and application deployments.
- Automate build, test, deployment, and monitoring processes.
- Manage Infrastructure as Code (IaC) using Terraform, CloudFormation, or similar tools.
- Improve deployment reliability and observability.

AI & Advanced Analytics

- Enable AI/ML workflows by building robust feature and training data pipelines.
- Work with AI teams to operationalize machine learning models.
- Explore Generative AI, LLM integrations, vector databases, and AI-driven automation solutions.
- Support model monitoring, governance, and deployment pipelines.

Leadership & Mentoring

- Provide technical leadership across multiple engineering teams.
- Mentor senior engineers and establish engineering standards.
- Collaborate with architects, product managers, and business stakeholders.
- Drive innovation and strategic technology initiatives.

📌 Principal Software Engineer (Data Engineering) (Chennai)
🏢 Nielseniq
📍 Chennai