13 Aug
|
Datapelago
|
Hyderabad
13 Aug
Datapelago
Hyderabad
System Software Engineer
Hyderabad, IN / Remote
About DataPelago:
DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an cutting-edge data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for specialists to join our engineering team and shape the future of accelerated data processing.
The Opportunity:
As a System Software Engineer, you will be a key individual contributor in advancing operating
system level capabilities underlying DataPelago’s data processing engine. You will enhance functional breadth, performance, scale, and reliability of these OS-level services on diverse accelerated hardware platforms.
This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.
What You'll Do:
• Architect: Influence the architecture of how our data processing engine harnesses and man-
ages the use of compute, memory, storage, and network resources.
• Design: Lead design of functional and performance enhancements to OS-level capabilities
underlying our data processing engine such as concurrent event processing, memory management, and inter-process communication.
• Core Development: Individually design, implement, test, optimize, and maintain components of the data processing engine.
• Innovation and Differentiation: Analyze advances in system services provided by platforms
and programming models for high-concurrency data processing on CPUs and GPUs (e.g.,
Rust, CUDA runtime) and identify opportunities for our engine to enhance technology and
product leadership.
• Collaboration: Partner effectively with
📌 System Software Engineer (Hyderabad)
🏢 Datapelago
📍 Hyderabad