A High-Performance Storage Engine for Modern Hardware
LeanStore is a high-performance OLTP storage engine optimized for many-core CPUs and NVMe SSDs. Our goal is to achieve performance comparable to in-memory systems when the data set fits into RAM, while being able to fully exploit the bandwidth of fast NVMe SSDs for large data sets. While LeanStore is currently a research prototype, we hope to make it usable in production in the future.
Highlights
Support for Very Large Data Sets on Directly-Attached NVMe Arrays
NVMe SSDs have become cheap (1 TB cost around 200 USD) and fast (easily achieving 5 GB/s bandwidth per device). Existing storage engines are not capable of exploiting such fast IO devices, in particular when multiple NVMe SSDs are combined in a single server. LeanStore has been designed from scratch for Directly-Attached NVMe Arrays to get the full performance from fast flash storage devices.
High In-Memory Performance
Using optimized index structures, LeanStore achieves very high in-memory performance and scalable synchronization techniques ensure that it scales very well on many-core CPUs. A lightweight buffer manager transparently keeps frequently-accessed data in RAM, while supporting arbitrarily-large data sets on SSD.
Embeddable Open Source Library
Similar to storage engines like RocksDB and WiredTiger, LeanStore can be embedded into applications by linking it as a library. It offers a C++ interface for basic data operations like point lookup, range lookup, insert, update, delete. Support for ACID transactions is currently under development. The source code is available under MIT license.
Team
Prof. Viktor Leis, Adnan Alhomssi, Gabriel Haas, Bo-Hyun Lee
Collaborators: Prof. Thomas Neumann, Prof. Alfons Kemper, Michael Haubenschild, Demian Vöhringer
Publications
| Venue | Publication | Link |
|---|---|---|
| VLDB 2024 | LeanStore: A High-Performance Storage Engine for NVMe SSDs Viktor Leis | |
| ICDE 2024 | Why Files If You Have a DBMS? Lam-Duy Nguyen, Viktor Leis | pdf - code |
| VLDB 2023 | Write-Aware Timestamp Tracking: Effective and Efficient Page Replacement for Modern Hardware Demian Vöhringer, Viktor Leis | pdf - code |
| SiMOD 2023 | Lock-Free Buffer Managers Do Not Require Delayed Memory Reclamation Michael Haubenschild, Viktor Leis | |
| VLDB 2023 | What Modern NVMe Storage Can Do, And How To Exploit It: High-Performance I/O for High-Performance Storage Engines Gabriel Haas, Viktor Leis | pdf - code |
| Scalable and Robust Snapshot Isolation for High-Performance Storage Engines Adnan Alhomssi, Viktor Leis | pdf - code | |
| BTW 2023 | The Evolution of LeanStore Adnan Alhomssi, Michael Haubenschild, Viktor Leis | pdf - code |
| SIGMOD 2023 | Virtual-Memory Assisted Buffer Management Viktor Leis, Adnan Alhomssi, Tobias Ziegler, Yannick Loeck, Christian Dietrich | pdf - code |
| CIDR 2021 | Contention and Space Management in B-Trees Adnan Alhomssi, Viktor Leis | pdf - code |
| SIGMOD 2020 | Rethinking Logging, Checkpoints, and Recovery for High-Performance Storage Engines Michael Haubenschild, Caetano Sauer, Thomas Neumann, Viktor Leis | |
| CIDR 2020 | Exploiting Directly-Attached NVMe Arrays in DBMS Gabriel Haas, Michael Haubenschild, Viktor Leis | |
| IEEE Data Engineering Bulletin | Optimistic Lock Coupling: A Scalable and Efficient General-Purpose Synchronization Method Viktor Leis, Michael Haubenschild, Thomas Neumann | |
| ICDE 2018 | LeanStore: In-Memory Data Management Beyond Main Memory Viktor Leis, Michael Haubenschild, Alfons Kemper, Thomas Neumann |