09 Oct
|
InFoCusp Innovations
|
Pune
09 Oct
InFoCusp Innovations
Pune
About the Role
We are seeking a highly skilled Senior Data Engineer with strong hands-on experience in Google Cloud Platform (GCP) or AWS, cloud infrastructure, data ingestion, and Python. The ideal candidate is also robust in dbt, Databricks, and AWS, and has experience implementing Medallion architecture. The ideal candidate will have a proven track record of designing and implementing scalable data platforms, organizing GCP or AWS environments to meet data sovereignty requirements, and building secure and reliable data ingestion and transfer solutions. They should be comfortable working with existing, imperfect codebases and improving them.
Key Responsibilities
1. GCP and AWS Architecture & Infrastructure
• Design, develop, and maintain scalable GCP infrastructure for enterprise data workloads.
• Architect and organize GCP environments to support data sovereignty and data residency requirements.
• Design appropriate GCP projects, environments, networking, storage, IAM, and security configurations for data platforms.
• Establish scalable and reliable cloud infrastructure patterns for data processing and storage.
• Work across GCP and AWS environments, applying consistent infrastructure and security patterns.
2. Data Ingestion & Pipeline Engineering
• Design and implement robust data ingestion pipelines for structured and unstructured data sources.
• Build production-grade ingestion frameworks to reliably process and move data into GCP or AWS, or similar storage.
• Develop scalable and fault-tolerant data pipelines with appropriate validation, monitoring, and error handling.
• Optimize data ingestion workflows for performance, reliability, and cost.
• Ensure data pipelines maintain data quality, integrity, and traceability throughout the ingestion process.
• Design and implement Medallion architecture (Bronze, Silver, Gold layers) to deliver clean, reliable, analytics-ready data.
• Build and maintain transformation workflows using dbt, including modeling, testing
📌 Senior Data Engineer (Pune)
🏢 InFoCusp Innovations
📍 Pune