19 Nov
|
pulsecrafts solutions
|
Bengaluru
19 Nov
pulsecrafts solutions
Bengaluru
Job Title: Data Engineer – Data Platforms (Google)
Company: IBM
Job Category: Data Engineering / Cloud Data Platforms
Experience: 5–10 Years (Versatile based on expertise)
Employment Type: Full-Time
Location: As per IBM project requirements
Job Overview
IBM is looking for an experienced Data Engineer – Data Platforms (Google Cloud) to design, develop, implement, and maintain scalable data engineering solutions using Google Cloud Platform (GCP). The ideal candidate should have strong hands-on experience in building modern data pipelines, managing large-scale datasets, developing ETL/ELT workflows, and optimizing cloud-based data platforms.
The candidate will work closely with data architects, analysts, data scientists, and engineering teams to deliver reliable, scalable, and high-performance data solutions.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and data integration solutions on Google Cloud Platform.
- Build and optimize ETL/ELT workflows for processing structured and unstructured data.
- Work extensively with Google BigQuery for data warehousing, querying, optimization, and performance tuning.
- Develop data processing solutions using Python, SQL, and PySpark.
- Implement and manage data pipelines using Cloud Dataflow, Apache Beam, and Cloud Composer (Airflow).
- Work with Google Cloud services such as Cloud Storage, Pub/Sub, Dataproc, and BigQuery.
- Develop data ingestion frameworks to integrate data from multiple internal and external sources.
- Implement data quality checks, data validation, monitoring, and error-handling mechanisms.
- Optimize data processing performance, storage, and cloud infrastructure costs.
- Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
- Ensure data security, governance, access controls, and compliance standards.
- Troubleshoot production data pipeline issues and ensure platform reliability.
Required Technical Skills
- Strong hands-on experience in Google Cloud Platform (GCP).
- Expertise in BigQuery, including complex SQL queries and performance optimization.
- Strong programming experience in Python and SQL.
- Experience developing ETL/ELT pipelines and data integration frameworks.
- Hands-on experience with Apache Beam, Dataflow, or PySpark.
- Experience with Cloud Storage, Pub/Sub, and Dataproc.
- Knowledge of workflow orchestration tools such as Cloud Composer or Apache Airflow.
- Understanding of data warehousing, data lakes, and modern data lakehouse architectures.
- Experience with data modeling, database design, and distributed data processing.
- Familiarity with Git, CI/CD, and DevOps practices.
Preferred Skills
- Google Cloud Professional Data Engineer certification.
- Experience with Terraform or Infrastructure as Code (IaC).
- Knowledge of Docker and Kubernetes.
- Experience with data governance, metadata management, and data catalog solutions.
- Familiarity with streaming technologies and real-time data processing.
- Experience working in Agile/Scrum environments.
Education Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related discipline.
Ideal Candidate Profile
- Strong problem-solving and analytical skills.
- Ability to design and deliver scalable enterprise-level data solutions.
- Excellent communication and collaboration skills.
- Ability to work independently and coordinate with global engineering teams.
- Strong understanding of modern cloud data engineering architectures.
Keywords for Sourcing Google Cloud Platform (GCP), BigQuery, Data Engineer, Dataflow, Apache Beam, Python, SQL, PySpark, Cloud Composer, Airflow, Pub/Sub, Dataproc, Cloud Storage, ETL, ELT, Data Pipelines, Data Warehousing, Data Lake, Terraform.
Pay: ₹1,000,000.00 - ₹1,200,000.00 per year
Benefits
- Provident Fund
Work Location: In person
📌 Data Engineer - Data Platforms-Google (Bengaluru)
🏢 pulsecrafts solutions
📍 Bengaluru