About the Role

We are looking for a Data Engineer who can manage data pipelines, integrations, data workflows, and orchestration using tools such as Kestra. The ideal candidate will be responsible for ensuring that data moves reliably between different systems, is properly transformed and validated, and is available for analytics, reporting, AI applications, and business operations.

This role requires someone who is comfortable working with databases, APIs, ETL/ELT processes, workflow orchestration, and cloud-based data infrastructure.

Key Responsibilities

- Design, build, maintain, and monitor data pipelines using Kestra and other ETL/ELT tools.
- Create and manage scheduled, event-driven, and automated workflows.
- Integrate data from APIs, databases, SaaS platforms, files, and other external sources.
- Perform data extraction, transformation, validation, and loading.
- Manage data across different databases and systems.
- Monitor pipeline execution, identify failures, troubleshoot issues, and ensure timely resolution.
- Implement error handling, retries, logging, alerts, and pipeline monitoring.
- Ensure data accuracy, consistency, completeness, and reliability.
- Develop SQL queries for data extraction, transformation, validation, and analysis.
- Work with relational and non-relational databases.
- Build reusable and scalable data workflows.
- Manage data ingestion from third-party APIs and external systems.
- Document pipelines, workflows, integrations, dependencies, and data sources.
- Optimize existing pipelines for performance, reliability, and cost.
- Collaborate with software developers,



BI/analytics teams, product teams, and business stakeholders.
- Support data requirements for dashboards, reporting, automation, and AI/ML applications.
- Maintain appropriate data security, access controls, and governance practices.

Required Skills
- Strong knowledge of SQL and relational databases.
- Hands-on experience with Kestra or similar workflow orchestration platforms.
- Understanding of ETL/ELT concepts and data pipelines.
- Experience working with REST APIs and third-party integrations.
- Experience with databases such as PostgreSQL, MySQL, SQL Server, or similar.
- Positive understanding of JSON, CSV, XML, and other common data formats.
- Experience with Python for data processing and automation.
- Understanding of data validation, transformation, and quality checks.
- Familiarity with Git and basic software development practices.
- Understanding of scheduling, dependencies, retries, logging, and monitoring of data workflows.
- Good troubleshooting and problem-solving skills.

Good to Have
- Experience with AWS or other cloud platforms.
- Experience with Docker and containerized workloads.
- Experience with data warehouses such as BigQuery, Snowflake, Redshift, or similar.




- Experience with BI tools such as Power BI, Looker, or Metabase.
- Experience with Kafka or other event-streaming technologies.
- Familiarity with Airflow, Dagster, Prefect, n8n, or similar orchestration platforms.
- Experience working with AI/ML data pipelines.
- Understanding of data governance and data security.
- Experience building monitoring and alerting systems for production pipelines.

What We Are Looking For The ideal candidate is someone who doesnt just build pipelines but takes ownership of the entire data flow. You should be comfortable asking:

- Where does the data come from
- Is the data complete and accurate
- How should it be transformed
- Where should it be stored
- What happens if the pipeline fails
- How do we know the pipeline is working
- How can the workflow be made more reliable and scalable

Strong attention to detail, analytical thinking, and the ability to troubleshoot production data issues are essential.

Experience

2-5 years of experience in Data Engineering, Data Operations, ETL/ELT, or a similar role.

Candidates with strong hands-on experience in Kestra and data pipeline management may be considered even if their overall experience is outside the traditional Data Engineering path.

", "

Disclaimer: This job posting & Location has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Data Engineer - Data Pipelines (India)
🏢 Aventus Informatics
📍 India