24 Oct
|
Costco IT
|
Hyderabad
24 Oct
Costco IT
Hyderabad
About Costco Wholesale: Costco Wholesale is a multi-billion-dollar global retailer with warehouse club operations in eleven countries. They provide a wide selection of quality merchandise, plus the convenience of specialty departments and exclusive member services, all designed to make shopping a pleasurable experience for their members. About Costco Wholesale India: At Costco Wholesale India, we foster a cooperative space, working to support Costco Wholesale in developing innovative solutions that improve members’ experiences and make employees’ jobs easier. Our employees play a key role in driving and delivering innovation to establish IT as a core competitive advantage for Costco Wholesale. We are seeking a proactive Associate/Mid-level GCP Data Engineer with 2 to 5 years of experience to design, build, and maintain scalable batch and streaming ETL/ELT pipelines on Google Cloud Platform. This role supports Retail analytics and Adtech/Martech data operations (including Retail Media Networks), working closely with senior engineers and analysts.
Key Responsibilities
• Design, build, and maintain batch and streaming data pipelines using Python, SQL, and PySpark on GCP.
• Orchestrate pipelines end-to-end using Cloud Composer (Apache Airflow), including DAG development, scheduling, dependency management, retries, alerting, and basic monitoring.
• Perform data ingestion from diverse sources (APIs, databases, ad logs, POS systems) and apply cleansing, transformation, and data quality logic.
• Work with core GCP services: BigQuery, Cloud Storage (GCS), Dataflow/Dataproc, Pub/Sub, and Cloud Composer.
• Support Retail and Adtech/Martech workflows such as ad impression logs, campaign performance tracking, customer segmentation,
and point-of-sale data integration.
• Implement data quality checks, basic data contracts, automated testing, logging, alerting, and error monitoring.
• Manage version control with GitHub and contribute to CI/CD pipeline deployments.
• Apply basic Cloud FinOps practices to monitor resource utilization and support cost optimization.
• Follow SRE fundamentals (availability, logging, alerting, error budgets) for data pipelines.
• Assume ownership and ongoing operational support for mission-critical data pipelines. Required Qualifications &
Skills:
• Up to 5 years of hands-on data engineering experience.
• Shell Scripting / Familiarity with gCloud SDK utilities.
• Solid grasp of data warehousing principles, data loading anti-patterns, and architectural trade-offs.
• Proficient in Python, SQL, and Spark ( Python / Scala APIs ).
• Working knowledge of GCP data products (BigQuery, GCS, Dataflow/Dataproc, Pub/Sub).
• Hands-on experience with pipeline orchestration tools (Cloud Composer / Apache Airflow) or equivalent..
• Familiarity with code versioning systems e.g. Github/Gitlab/Bitbucket/, CI/CD concepts, and basic data quality practices.
• Proven analytical background, keen attention to detail, strong problem-solving capabilities, and experience collaborating with cross-functional, global teams. Good to Have:
• Exposure to DBT for analytics engineering.
• Basic understanding of Retail domain data and Adtech/Martech processes.
• Experience with data quality/observability tools or basic data contracts.
• Familiarity with Infrastructure-as-Code (Terraform), automation skills.
• Basic exposure AI Tools driven development, understanding of Skills / agent.
📌 Data Engineer Level 3 (Hyderabad)
🏢 Costco IT
📍 Hyderabad