- Strong Python programming skills, including experience developing production data pipelines and integrating with APIs.
- Proficiency in SQL for data loading, validation, and troubleshooting.
- Hands-on experience with Snowflake and relevant Azure data and storage services.
- Experience building and operating workflows with Prefect or a comparable orchestration tool, with the ability to work in Prefect.
- Experience ingesting structured and semi-structured data, including formats such as CSV, JSON, and Parquet.
- Understanding of batch and incremental ingestion patterns, pagination, API rate limits, and schema evolution.
- Experience using Git, code reviews, and automated testing.
- Solid troubleshooting, documentation, and communication skills.
- Ability to work independently while coordinating across internal teams and external organizations.

Expected outcomes

- Reliable, timely ingestion of source data into the raw data layer.
- Maintainable pipelines with clear monitoring, validation, and recovery procedures.
- Documented integrations that can be supported and extended by other engineers.
- Prompt identification and resolution of ingestion issues.

- Develop and maintain data ingestion pipelines using Python, Prefect, Azure, and Snowflake.
- Integrate data from enterprise applications, SaaS tools, and external partners through APIs, database connections,



and file transfers.
- Land raw data in designated storage and Snowflake environments, preserving source fidelity and traceability.
- Implement scheduled and incremental ingestion, including retries, error handling, duplicate prevention, and recovery from failed runs.
- Monitor pipeline execution, troubleshoot issues, and resolve data delivery failures.
- Implement ingestion checks for completeness, freshness, schema changes, and reconciliation with source systems.
- Collaborate with business teams, application owners, and external partners to define source interfaces, data formats, delivery schedules, and access requirements.
- Follow established standards for secure access, credential management, logging, and handling sensitive data.
- Maintain pipeline documentation, source mappings, operational runbooks, and support handover materials.

- Strong Python programming skills, including experience developing production data pipelines and integrating with APIs.
- Proficiency in

... Data engineer, Claude

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Data Engineer (Bengaluru)
🏢 Infovision labs
📍 Bengaluru