1. How much percentage you spend writing Python script for data cleanup, or any other work
2. Big Query - which tool you have used?
3. Have you worked on Batch processing of data or Streaming? How much exp: Batch processing, How much on Streaming
4. Have you worked on workflow creation using Airflow? Yes/No
5. SQL rating? then Have you worked on SQL query tuning? Yes/No
6. Which domain you have worked? Note down the domain
7. Have to created Data pipeling for ETL end to end? years of experience?
Job Description
Minimum 12 months of core experience in Airflow (writing
Directed Acyclic Graphs DAGs) Good
to have
Minimum 12 months of core experience in Big Query,
Composer, and development of pipelines (batch or streaming) - MUST
Minimum 6 months of experience in Python - MUST
Very strong in SQL (at least 4 years of experience in
writing SQLs) - MUST
Solid communication skills - MUST
Knowledge in Pub/Sub would be an advantage - Positive To Have
Knowledge of the telecom domain would be an advantage - Good To Have
- Have experience with Data Fusion or Equivalent, Big Query or Equivalent, SQL server, scripting in Java/Python that works well in GCP products, and their respective practices. Python experience is a plus.
- Develop data ETL pipelines that meet both functional and non-functional requirements, including performance, scalability, availability, reliability, and security.
- Have experience with writing code in Java, to work on data extracts that require cleanup.
- Have a working knowledge of XML, JSON, and other forms of data streaming artifacts and related technologies in a Java/Python environment.
- Have strong written and verbal communication skills
- Able to multi-task on various streams of the entire data process.