16 Aug
|
Diverse Lynx India Private
|
Gurugram
16 Aug
Diverse Lynx India Private
Gurugram
Please Apply those candidates who are immediate joiner with relevant 5+ years of exp in all 3 skills.
GCP Data Engineer:
Total Experience – 6 years
Relevant exp – 5 years
Location: Chennai And Gurugram
Mandatory Skill : GCP data Engineer with Python ,Pyspark
Please share your resume at : (HIDDEN TEXT)
Job Summary
We are looking for a skilled Data Engineer with strong expertise in Python, PySpark, and Google Cloud Platform (GCP) to design, build, and optimize scalable data pipelines and data processing systems.
The ideal candidate should have hands-on experience working with big data technologies , ETL pipelines , and cloud-based data platforms , with a strong understanding of data modeling and distributed processing.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Python and PySpark
- Build and optimize ETL/ELT processes for large-scale data processing
- Work with GCP services such as BigQuery, Dataflow, Dataproc, Cloud Storage
- Develop and manage batch and real-time data pipelines
- Ensure data quality, integrity, and security across pipelines
- Optimize data processing jobs for performance and cost efficiency
- Collaborate with data analysts, data scientists, and business teams to understand requirements
- Implement data warehousing solutions and support reporting needs
- Monitor pipeline performance and troubleshoot issues
- Work with Airflow/Cloud Composer for workflow orchestration
- Participate in code reviews, testing, and deployment activities
Technical Skills
Core Skills:
- Robust programming experience in Python
- Hands-on experience with PySpark / Apache Spark
- Strong knowledge of ETL / Data Pipelines
GCP (Must Have):
- BigQuery
- Dataflow
- Dataproc
- Cloud Storage
- Cloud Composer (Airflow)
Data Technologies:
- SQL (advanced queries, optimization)
- Data Warehousing concepts
- Batch and streaming data processing
Tools / Ecosystem:
- Git / GitHub
- CI/CD pipelines
- Docker / Kubernetes (nice to have)
Good to Have
📌 GCP Data Engineer (Gurugram)
🏢 Diverse Lynx India Private
📍 Gurugram