Job Summary

Lead PySpark Cloud Data Engineer: This job involves understand the overall requirement of the enterprise Data need and design develop the robust, highly scalable resilient data pipeline using PySpark, Dataproc, open source table formats and other Google services. Job will involve extensive interfacing co-ordination with other senior tech folks (across India US) and lead the design development of the data ingestion pipelines.

The experience we are looking to add to our team

Required
Bachelors degree or higher from a reputed university
8 to 12 years total experience with majority of that experience related to building high performant data pipelines batch and streaming
Solid Hands on experience / expert-level proficiency in PySpark for data processing, transformation, and analysis
Extensive experience in implementing large scale data ingestion and curation solutions
Hands-on experience with cloud-based data platforms, preferably Google Cloud Platform (GCP) and services like Dataproc, BigQuery, and Cloud Storage




Knowledge in Apache Iceberg for open table formats
Advanced SQL skills for data querying, manipulation, and optimization
Proficiency with Git and collaborative development workflows
Excellent analytical and problem-solving skills with a keen attention to detail
Solid communication and interpersonal skills, with the ability to explain complex technical concepts to both technical and non-technical audiences

Good to have
Expertise in Google Cloud services
Proven experience with Apache Iceberg for open table formats and BigLake Metastore for unified metadata management
Robust experience with dbt (data build tool) for data modeling, transformation, and orchestration
Experience in Data Goverce and Data quality tools like Atlan, Monte Carlo etc.
Experience in processing streaming data using Kafka / Pub-Sub
Healthcare industry experience
Experience in Agile

📌 Senior Consultant Lead Pyspark Cloud Data Engineer Chennai (India)
🏢 enGen Global
📍 India