17 Nov
|
Tekskills
|
Noida
Job highlights
Expertise in Apache Spark (batch + streaming), Hive, Python/Scala/Java, orchestration tools (Airflow/Control-M), Kafka, object stores (S3, MinIO), and data Lakehouse formats (Iceberg, Delta Lake, Hudi)
Develop and manage data pipelines using Spark and Hive; implement orchestration with Airflow/Control-M; work with Kafka and object stores; deploy using Docker/Kubernetes
Job Summary
Role &
responsibilities Solid expertise in Apache Spark (batch + streaming) and Hive. Proficiency in Python, Scala, or Java. Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
Experience working with Kafka, Solace, and object stores (S3, MinIO). Exposure to Docker/Kubernetes for deployment.
Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).
Responsibilities
Solid expertise in Apache Spark (batch + streaming) and Hive.
Proficiency in Python, Scala, or Java.
Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
Experience working with Kafka, Solace, and object stores (S3, MinIO).
Exposure to Docker/Kubernetes for deployment.
Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).
📌 Data Engineer Pyspark Noida
🏢 Tekskills
📍 Noida