HEC: Collaborative Research: Techniques for Streaming File Systems and Databases

The performance of many high-end computing applications is limited by the capacity of memory systems to deliver data. As processor speeds increase, I/O performance continues to lag substantially. Thus, I/O is likely to remain a critical bottleneck for high-end computing. In this research project The PIs propose to build a prototype of a streaming B-tree for deployment in a database or file system. This prototype will realize t fast insertions, fast range queries, and platform independence. The PIs will simultaneously work on how to deal with different-sized keys, how to support transactions, how to scale to multiple disks and processes, and how to provide O/S support for cache-obliviousness and memory-mapped massive data. The proposed work represents a promising new direction in how to manipulate massive data and overcome classic I/O bottlenecks.