Build scalable data and feature engineering pipelines powering the anomaly detection models on BigQuery and Python.
Key Responsibilities
Develop BigQuery + SQL/Python pipelines for ingestion and transformation. · Perform schema mapping across Entra ID and TrendMicro datasets. · Implement feature engineering for ML model consumption. · Optimize pipeline performance, cost, and reliability on GCP. · Partner with the ML Engineer for model-ready datasets.
Skill Requirements
Advanced Python, SQL, and BigQuery skills. · Strong in ETL/ELT, data modeling, and schema design. · Experience with GCP data services (Dataflow, Pub/Sub, Cloud Composer). · Knowledge of feature stores and pipeline orchestration (Airflow). · Understanding of security/identity data structures.
Other Requirements
Education: Bachelor’s degree in IT, Computer Science, or equivalent qualified experience. Experience: 10–15 years