Responsibilities

- What you'll do Engineer, test, document and manage GCP Dataproc, DataFlow and VertexAI services used in high-performance data processing pipelines and Machine Learning.
- Help developers optimize data processing jobs using Spark, Python, and Java.
- Collaborate with development teams to integrate data processing pipelines with other cloud services and applications.
- Utilize Terraform and Tekton for infrastructure as code (IaC) and CI/CD pipelines, ensuring efficient deployment and management.

Valuable to have

- Experience with Spark for large-scale data processing.
- Solid understanding and experience with GitHub for version control and collaboration.
- Experience with Terraform for infrastructure management and Tekton for continuous integration and deployment.
- Experience with Apache NiFi for data flow automation.
- Knowledge of Apache Kafka for real-time data streaming.
- Familiarity with Google Cloud Pub/Sub for event-driven systems and messaging. Familiarity with Google BigQuery

📌 Systems Engineering Practitioner (Pune)
🏢 Shashwath Solution
📍 Pune