We are looking for a skilled GCP Data Engineer with strong hands-on experience in Python, SQL, and Google Cloud Platform. The candidate will be responsible for designing, developing, and maintaining scalable data pipelines and data-processing solutions on GCP.
Key Responsibilities
- Design and develop scalable data pipelines using Python and GCP services.
- Develop and optimize complex SQL queries for data extraction, transformation, and analysis.
- Build and maintain ETL/ELT workflows for enterprise data platforms.
- Work with GCP data services such as BigQuery, Cloud Storage, Dataflow, Pub/Sub, and Cloud Composer.
- Perform data ingestion, transformation, validation, and quality checks.
- Integrate data from databases, APIs, files, and other source systems.
- Optimize data pipelines and queries for performance and cost efficiency.
- Troubleshoot pipeline failures and resolve data-quality and production issues.
- Implement data security, access controls, and best practices within GCP environments.
- Collaborate with Data Architects, Developers,
Analysts, and other stakeholders.
- Use Git and follow CI/CD and Agile development practices.
- Prepare technical documentation for data pipelines and solutions.
Must-Have Skills
- 3–12 years of experience in Data Engineering.
- Strong programming experience in Python.
- Strong hands-on experience with SQL.
- Experience working with Google Cloud Platform (GCP).
- Hands-on experience with BigQuery.
- Good understanding of ETL/ELT and data warehousing concepts.
- Experience developing and troubleshooting data pipelines.
- Good knowledge of data structures, data processing, and database concepts.
- Experience with Git/version control.
- Strong analytical and problem-solving skills.
Valuable to Have
- Cloud Dataflow
- Cloud Composer / Apache Airflow
- Pub/Sub
- Cloud Storage (GCS)
- Dataproc / Spark
- Terraform or Infrastructure as Code
- CI/CD
- GCP IAM and security concepts
- Exposure to Kafka or other streaming technologies