Dagstuhl-Seminar 24162
Hardware Support for Cloud Database Systems in the Post-Moore’s Law Era
( 14. Apr – 19. Apr, 2024 )
Permalink
Organisatoren
- David F. Bacon (Google - New York, US)
- Carsten Binnig (TU Darmstadt, DE)
- David A. Patterson (University of California - Berkeley, US)
- Margo Seltzer (University of British Columbia - Vancouver, CA)
Kontakt
- Andreas Dolzmann (für wissenschaftliche Fragen)
- Jutka Gasiorowski (für administrative Fragen)
Gemeinsame Dokumente
- Dagstuhl Materials Page (Use personal credentials as created in DOOR to log in)
Programm
This Dagstuhl Seminar on the Future of Cloud Database Systems was convened to address the pressing challenges arising from the stagnation in hardware performance gains, historically driven by Moore’s and Dennard’s laws. As data continues to grow exponentially – propelled by the expansion of autonomous systems, the Internet of Things (IoT), and machine learning – there is an urgent need to rethink the co-design of database systems and hardware. This seminar brought together experts from database systems, hardware architecture, and storage systems to explore innovative approaches to overcoming these scalability bottlenecks and envisioning the future of cloud database systems.
A central theme of the seminar was the growing disconnect between the exponential increase in data and the slowing pace of hardware improvements, leading to what participants referred to as a “scalability wall.” Addressing this challenge requires groundbreaking architectural changes in cloud database systems to support the next generation of applications. One significant area of focus was the potential role of AI-driven hardware and software in reshaping database management systems (DBMS). Participants explored whether AI hardware, such as GPUs and TPUs, could be adapted for database workloads, which traditionally are not compute-bound. Additionally, the concept of leveraging large language models (LLMs) as a new paradigm for databases was discussed, prompting further considerations of the future interplay between AI and DBMS.
To kickstart these discussions, several invited impulse talks were presented, each designed to set the stage for the working groups by exploring possible future scenarios for cloud database systems:
- AI Rules: This talk examined a future where AI hardware and software dominate data centers, fundamentally altering the design and function of DBMS. The discussion centered on how DBMSs might need to evolve in a world where AI is integral to data processing and whether an LLM could serve as a database.
- A Disaggregated Future: This presentations offered a perspective on a future where heterogeneous devices (compute, memory, storage) are connected via ultra-fast networks, creating a fully disaggregated cloud infrastructure. The talk prompted discussions on how DBMS could adapt to and thrive in such an environment.
- A Fully Reprogrammable Future: The talk on this future envisioned a future where all hardware is reprogrammable and customizable at runtime, drastically changing how data processing and storage are handled. The implications for DBMS in such a highly flexible hardware environment were critically examined.
- The Pipe Dream: This session explored the idea of "dreaming up" new DBMS hardware, revisiting the concept of a dedicated database machine. The discussion focused on whether this approach, which has failed in the past, could succeed in the context of modern cloud environments.
Following these impulse talks, the seminar divided into working groups to delve deeper into specific challenges:
- Working Group 1: The Next Order of Magnitude focused on how database technologies can evolve to achieve order-of-magnitude improvements in performance, despite the slowdown in hardware advancements. This group was particularly concerned with managing the exponential growth of unstructured data feeding machine learning models.
- Working Group 2: Memory-Centric DBMS Design advocated for a shift from processor-centric to memory-centric designs, emphasizing the optimization of data access in cloud environments as a solution to the performance bottlenecks caused by traditional architectural models.
- Working Group 3: AI Hardware for Databases investigated how emerging AI hardware, like GPUs and TPUs, could be leveraged for cloud DBMS, even though database workloads typically do not benefit as much from compute-bound acceleration as other applications do.
- Working Group 4: The last working group explored taking disaggregation to the extreme and considering its impact on systems for cloud DBMSs.
As the seminar progressed, participants emphasized the importance of cross-disciplinary collaboration and knowledge sharing. They worked together to draft a comprehensive paper for publication, summarizing the insights and innovations discussed. The seminar concluded with a focus on the need for continued innovation in both hardware and software to meet the demands of future cloud database systems.
In summary, the Dagstuhl Seminar provided a crucial platform for reimagining the future of cloud database systems in light of hardware stagnation. By bringing together leading experts from multiple disciplines and sparking deep discussions through targeted impulse talks, the seminar laid the groundwork for the architectural and system-level innovations necessary to overcome the scalability challenges posed by exponential data growth. The insights and collaborative efforts from this seminar will be instrumental in guiding the development of next-generation database systems.

The end of scaling from Moore’s and Dennard’s laws has greatly slowed improvements in CPU speed, RAM capacity, and disk/flash capacity. Meanwhile, cloud database systems, which are the backbone for many large-scale services and applications in the cloud, are continuing to grow exponentially. For example, most of Google’s products that run on the Spanner database have more than a billion users and are continuously growing. Moreover, the growth in data also shows no signs of slowing down, with further orders-of-magnitude increases likely, due to autonomous vehicles, the internet-of-things, and human-driven data creation. Meanwhile, machine learning creates an appetite for data that also needs to be pre-processed using scalable cloud database systems. As a result, cloud database systems are facing a fundamental scalability wall on how to further support this exponential growth given the stagnation in hardware.
While database research has a long tradition of investigating how modern hardware can be leveraged to improve overall system performance – which is also shown by the series of past Dagstuhl Seminars – a more holistic view is required to address the imminent exponential scalability challenge that databases will be facing. However, applying hardware accelerators in database needs a careful design. In fact, so far, no commercial system has applied hardware accelerators at scale. Unlike other hyper-scale applications such as machine learning training and video processing where accelerators such as GPUs and TPUs circumvent this problem, workloads in cloud database systems are typically not compute-bound and thus benefit less or not at all from such existing accelerators. Database systems also rely more on balanced performance across the full storage hierarchy, from level 1 caches all the way to disks accessed over the network.
This Dagstuhl Seminar aim to bringing together leading researchers and practitioners from database systems, hardware architecture, and storage systems to rethink, from the ground up, how to co-design database systems and compute/storage hardware. By bringing together experts across these disciplines, we hope to identify the architectural changes and system designs that will enable the order-of-magnitude improvements required for the next generation of applications. Many directions can be discussed: Instead of focusing on how to leverage accelerators, should we not focus on how to efficiently enable systems that allow us to have a more flexible balance of cores, accelerators, fabrics, RAM, and IO? In the same vein, we think that it is worth investigating how to customize CPU architectures for database workloads instead of using existing accelerators such as GPUs? Another interesting direction is how do we balance disaggregation which promotes resource efficiency with aggregation that promotes performance? And most prominently, how do we avoid the pitfalls of much past work, both academic and industrial, on accelerators or storage subsystems that wind up failing due to limited impact?

Please log in to DOOR to see more details.
- Anastasia Ailamaki (EPFL - Lausanne, CH) [dblp]
- Gustavo Alonso (ETH Zürich, CH) [dblp]
- David F. Bacon (Google - New York, US) [dblp]
- Lawrence Benson (TU München, DE) [dblp]
- Carsten Binnig (TU Darmstadt, DE) [dblp]
- Alexander Böhm (SAP SE - Walldorf, DE) [dblp]
- Helena Caminal (Google - Sunnyvale, US)
- Yannis Chronis (Google - Sunnyvale, US) [dblp]
- Holger Fröning (Universität Heidelberg - Mannheim, DE) [dblp]
- Jana Giceva (TU München - Garching, DE) [dblp]
- Mark D. Hill (University of Wisconsin-Madison, US) [dblp]
- Ihab Francis Ilyas (University of Waterloo, CA) [dblp]
- Zsolt Istvan (TU Darmstadt, DE) [dblp]
- Lana Josipovic (ETH Zürich, CH)
- Tim Kraska (MIT - Cambridge, US) [dblp]
- Justin Levandoski (Google - Seattle, US) [dblp]
- Jignesh M. Patel (Carnegie Mellon University - Pittsburgh, US) [dblp]
- David A. Patterson (University of California - Berkeley, US) [dblp]
- Holger Pirk (Imperial College London, GB) [dblp]
- Tilmann Rabl (Hasso-Plattner-Institut, Universität Potsdam, DE) [dblp]
- Eric Sedlar (Oracle Labs - Redwood Shores, US) [dblp]
- Margo Seltzer (University of British Columbia - Vancouver, CA) [dblp]
- Pinar Tözün (IT University of Copenhagen, DK) [dblp]
- Nandita Vijaykumar (University of Toronto, CA)
- Tianzheng Wang (Simon Fraser University - Burnaby, CA) [dblp]
- Lisa Wu Wills (Duke University - Durham, US)
- Tobias Ziegler (TU Darmstadt, DE) [dblp]
Klassifikation
- Databases
- Hardware Architecture
Schlagworte
- Database systems
- Hyperscale systems
- Cloud computing
- Computer architecture
- Hardware/software co-design