05 Oct
|
IntraEdge
|
Pune
Job Title: Data Engineer - AI/ML Engineer
Location: Remote/Pune
Job Type: Full Time
The shift timing: 2pm to 11pm
Responsibilities
• Design, build, and maintain scalable data pipelines to support analytics, ML, and operational reporting.
• Develop robust data ingestion, transformation, and integration workflows using Python, SQL, and contemporary data engineering frameworks.
• Build and maintain batch and streaming data pipelines leveraging technologies such as Kafka (or similar pub/sub tools).
• Work with Google Cloud Platform (GCP) services, including Cloud Storage, Dataflow, Pub/Sub, BigQuery, Cloud Spanner and Cloud Functions.
• Develop and manage data APIs and interfaces (REST and GraphQL) to enable high-performance data access across microservices.
• Implement CI/CD automation for data pipelines using GitHub Actions, Argo CD, or equivalent tools.
• Collaborate with Data Scientists and MLOps teams to integrate ML/NLP models into data pipelines and production workflows.
• Build and operationalize NLP data pipelines for structured and unstructured data sources (e.g.,
Rx claims, clinical documents).
• Enable continuous learning and model-retraining workflows using Vertex AI, Kubeflow, or similar GCP-native tooling.
• Implement frameworks for observability and data quality, ensuring ML predictions, confidence scores, and fallback events are logged into data lakes or monitoring systems.
• Support distributed data systems and ensure reliability, performance, and scalability of data infrastructure.
Infrastructure Engineering Responsibilities
• Design and provision cloud infrastructure using Infrastructure as Code (IaC) tools such as Terraform or Pulumi for GCP resources including GKE clusters, Cloud SQL, VPC networks, IAM, and storage.
• Deploy, configure, and manage containerized data workloads using Kubernetes (GKE) — including deployments, autoscaling (HPA/VPA), namespaces, resource quotas, and health checks.
• Architect and maintain network topology for data platform
📌 Data Engineer (Pune)
🏢 IntraEdge
📍 Pune