Senior Data Engineer Developer

Job Summary

We are looking for a Senior Data Engineer with 5-10 years of experience to design, build, and optimize scalable data pipelines that power analytics, reporting, and application-level data needs. This role requires strong hands-on development skills combined with the ability to make sound architectural and design decisions, and collaborate closely with data architects, analysts, and business stakeholders to deliver reliable, high-performance data solutions.

Key Responsibilities

1. Data Pipeline Development

- Design, build, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from diverse sources (databases, APIs, files, streaming systems).
- Develop batch and real-time data processing workflows using modern data engineering frameworks.
- Optimize pipelines for performance, reliability, scalability, and cost-efficiency.
- Implement robust error handling, logging, retry logic, and monitoring for all data workflows.

2. Data Quality & Governance

- Implement data validation, cleansing, and reconciliation processes to ensure data accuracy and consistency.
- Establish and enforce data quality checks and monitoring/alerting frameworks.
- Support data governance initiatives including metadata management, lineage tracking, and documentation.

3. Integration & Automation

- Build integrations between internal systems, third-party APIs, and cloud services.
- Automate manual data processes to improve operational efficiency.
- Work with orchestration tools (Airflow, Azure Data Factory, dbt, etc.)



to schedule and manage complex workflows.

4. Performance Optimization & Troubleshooting

- Tune SQL queries, pipeline jobs, and database performance for large-scale datasets.
- Troubleshoot and resolve data pipeline failures, latency issues, and data discrepancies.
- Conduct root cause analysis for recurring data issues and implement long-term fixes.

Required Qualifications

Experience

- 5-10 years of hands-on experience in data engineering, ETL development, or related backend/data roles.
- Proven experience designing and delivering production-grade data pipelines at scale.
- Experience working in Agile/Scrum development environments.

Technical Skills

- Solid proficiency in SQL (complex queries, performance tuning, stored procedures).
- Strong programming skills in Python, Scala, or Java for data processing.
- Hands-on experience with ETL/ELT tools (Informatica, Talend, SSIS, dbt, or similar).
- Experience with big data technologies (Spark, Hadoop, Kafka) is a strong plus.
- Proficiency with cloud data platforms (AWS Redshift/Glue, Azure Synapse/Data Factory, GCP BigQuery/Dataflow, Snowflake, or Databricks).
- Experience with orchestration/workflow tools (Airflow, Azure Data Factory, Prefect, Luigi).
- Solid understanding of relational and non-relational databases (PostgreSQL, MySQL, MongoDB, Cassandra).
- Familiarity with version control (Git) and CI/CD pipelines for data engineering workflows.
- Understanding of data modeling concepts (star schema, snowflake schema, normalization).

📌 Data Engineer (Chennai)
🏢 Ignitho
📍 Chennai