GCP Senior Data Engineer | 6-10 Years of experience |
n
Chennai/ Hyderabad / Kolkata /Bangalore
n
Required Technical Skills
n
n
- Python
n
- SQL
n
- PySpark
n
- GCP (Preferred) or Azure
n
- Snowflake
n
- Airflow
n
- Jenkins / Jira / Git
n
- CI/CD
n
- Kubernetes / Docker
n
n
Must-Have
n
n
- Strong development experience in Python, PySpark, and SQL.
n
- Experience with cloud platforms (GCP preferred).
n
- Experience working in an Agile delivery environment.
n
- Experience with orchestration tools such as Airflow and ADF.
n
- Experience with GCP services:
n
- GCS Bucket
n
- Cloud Functions
n
- Dataproc
n
- Dataflow
n
- Pub/Sub
n
- BigQuery
n
- Hands-on experience in Snowflake.
n
- Experience building automated data pipelines.
n
- Experience in data analysis and data exploration.
n
- Experience working in a multi-developer workplace using version control tools such as Git.
n
- Strong critical thinking, communication, and problem-solving skills.
n
n
Good-to-Have
n
n
- Understanding of DevOps best practices and CI/CD.
n
- Understanding of containerization technologies such as Kubernetes and Docker.
n
- Healthcare domain knowledge.
n
n
Key Responsibilities
n
n
- Perform data analytics on large-scale datasets to solve complex business problems and recommend solutions.
n
- Develop and enhance end-to-end data pipelines using big data technologies such as Hadoop, Cloud platforms, and the Spark ecosystem.
n
- Troubleshoot complex data structures and perform root cause analysis to proactively resolve operational and product issues.
n
- Develop and document test cases, execute testing, and share UAT results as part of Agile sprints.
n
- Perform end-to-end data pipeline integration testing in compliance with change management processes and standards.
n
- Adhere to coding, design, and application standards, including Python and SQL best practices.
n
- Leverage DevOps practices for production deployments, including CI/CD, test-driven development, and version control using Jenkins, Jira, and Git.
n
- Collaborate with business stakeholders to define product objectives, business requirements, system designs, and key performance indicators (KPIs).
n