Principal Data Processing Engineer - OSS
Hyderabad, IN

About DataPelago:

DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray, and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for world-class engineers to join our team and shape the future of accelerated data processing.

The Role:

As a Principal Data Processing Engineer (OSS), you will be a key individual contributor in
adopting and advancing the capabilities of open-source software (OSS) platforms such as Apache

Gluten, Velox, Apache Spark, and Apache Flink in the context of DataPelago’s data processing engine. You will enhance the functional breadth, performance, scale,



and reliability of the DataPelago engine through downstream and upstream contributions. You will have the opportunity to engage with the community working on these platforms. This is a unique prospect to make a significant impact on a category-defining product and work with a talented team of engineers.

What You'll Do:
- Influence the architecture of how our data processing engine interfaces with open-source platforms and engines.
- Lead the design of functional and performance enhancements to open source platforms such as Apache Gluten and Velox, and their integration with our data processing engine.
- Individually design, implement, test, optimize, and maintain components of the data processing engine.
- Analyze the technology roadmap of Apache Gluten, Velox, and equivalent platforms and identify opportunities for our engine to enhance technology and product leadership.
- Collaboration: Partner with engineering, product ma

📌 Principal Data Processing Engineer-OSS (Hyderabad)
🏢 Datapelago
📍 Hyderabad