Performance Evaluation of Main Memory Database Systems


Book Description

In this paper we present the results of a comprehensive benchmark of the relational Main Memory Database System (MMDBS), that is the foundation of the interactive office system. Office-By-Example (OBE). Based on this case study, we identify issues that must be considered in the design and implementation of MMDBS's. We determine relevant performance metrics and describe techniques for benchmarking MMDBS's.







Main Memory Databases


Book Description

"The declining cost of main memory coupled with higher performance expectations has inspired researchers and developers to look into systems with massive amounts of memory, especially for systems with databases. When memory is large enough, an entire database can permanently reside in main memory. These database systems are known as memory resident database or Main Memory DataBase (MMDB) systems. When a database is a main memory resident, systems can achieve significant performance improvements over traditional disk-based database systems by eliminating the overhead required to perform Input/Output operations. However, the problem of crash recovery becomes by far the most challenging issue in MMDB systems, since it can occur frequently and cause a drastic crisis by wiping out the entire content of the database. The kind of crisis that can be encountered due to memory failures, the way to prevent them from occuring in the first place, methods of dealing with them upon their occurance, strategies to make the system more reliable to avoid them in the future, and several other issues like these must be well understood. Approaches to deal with crash recoveries must be sound, well designed, and correctly implemented. The objective of this thesis is to study some of the pioneering works of researchers and developers in the area of MMDB recovery and to perform an in-depth analysis of MMDB recovery issues and approaches taken by several major recovery techniques. The analysis examines the reasons for these techniques, design and architecture, and their benefits and limitations over one another. My independent evaluation of the relative benefits and limitations of each technique coupled with marketablitiy analysis is also presented. The approach used in this study is different from that of other studies that have been published so far in this area. In this study, the reader is given a basic introduction to the topic and then gradually taken through the final stages of analysis and comparison of several recovery techniques."--Author's abstract.




Real-Time Database Systems


Book Description

Despite the growing interest in Real-Time Database Systems, there is no single book that acts as a reference to academics, professionals, and practitioners who wish to understand the issues involved in the design and development of RTDBS. Real-Time Database Systems: Issues and Applications fulfills this need. This book presents the spectrum of issues that may arise in various real-time database applications, the available solutions and technologies that may be used to address these issues, and the open problems that need to be tackled in the future. With rapid advances in this area, several concepts have been proposed without a widely accepted consensus on their definitions and implications. To address this need, the first chapter is an introduction to the key RTDBS concepts and definitions, which is followed by a survey of the state of the art in RTDBS research and practice. The remainder of the book consists of four sections: models and paradigms, applications and benchmarks, scheduling and concurrency control, and experimental systems. The chapters in each section are contributed by experts in the respective areas. Real-Time Database Systems: Issues and Applications is primarily intended for practicing engineers and researchers working in the growing area of real-time database systems. For practitioners, the book will provide a much needed bridge for technology transfer and continued education. For researchers, this book will provide a comprehensive reference for well-established results. This book can also be used in a senior or graduate level course on real-time systems, real-time database systems, and database systems or closely related courses.




Performance Evaluation: Origins and Directions


Book Description

This monograph-like state-of-the-art survey presents the history, the key ideas, the success stories, and future challenges of performance evaluation and demonstrates the impact of performance evaluation on a variety of different areas through case studies in a coherent and comprehensive way. Leading researchers in the field have contributed 19 cross-reviewed topical chapters competently covering the whole range of performance evaluation, from theoretical and methodological issues to applications in numerous other fields. Additionally, the book contains one contribution on the role of performance evaluation in industry and personal accounts of four pioneering researchers describing the genesis of breakthrough results. The book will become a valuable source of reference and indispensable reading for anybody active or interested in performance evaluation.




Non-Volatile Memory Database Management Systems


Book Description

This book explores the implications of non-volatile memory (NVM) for database management systems (DBMSs). The advent of NVM will fundamentally change the dichotomy between volatile memory and durable storage in DBMSs. These new NVM devices are almost as fast as volatile memory, but all writes to them are persistent even after power loss. Existing DBMSs are unable to take full advantage of this technology because their internal architectures are predicated on the assumption that memory is volatile. With NVM, many of the components of legacy DBMSs are unnecessary and will degrade the performance of data-intensive applications. We present the design and implementation of DBMS architectures that are explicitly tailored for NVM. The book focuses on three aspects of a DBMS: (1) logging and recovery, (2) storage and buffer management, and (3) indexing. First, we present a logging and recovery protocol that enables the DBMS to support near-instantaneous recovery. Second, we propose a storage engine architecture and buffer management policy that leverages the durability and byte-addressability properties of NVM to reduce data duplication and data migration. Third, the book presents the design of a range index tailored for NVM that is latch-free yet simple to implement. All together, the work described in this book illustrates that rethinking the fundamental algorithms and data structures employed in a DBMS for NVM improves performance and availability, reduces operational cost, and simplifies software development.




High Performance Memory Systems


Book Description

The State of Memory Technology Over the past decade there has been rapid growth in the speed of micropro cessors. CPU speeds are approximately doubling every eighteen months, while main memory speed doubles about every ten years. The International Tech nology Roadmap for Semiconductors (ITRS) study suggests that memory will remain on its current growth path. The ITRS short-and long-term targets indicate continued scaling improvements at about the current rate by 2016. This translates to bit densities increasing at two times every two years until the introduction of 8 gigabit dynamic random access memory (DRAM) chips, after which densities will increase four times every five years. A similar growth pattern is forecast for other high-density chip areas and high-performance logic (e.g., microprocessors and application specific inte grated circuits (ASICs)). In the future, molecular devices, 64 gigabit DRAMs and 28 GHz clock signals are targeted. Although densities continue to grow, we still do not see significant advances that will improve memory speed. These trends have created a problem that has been labeled the Memory Wall or Memory Gap.




Main Memory Database Systems


Book Description

With growing memory sizes and memory prices dropping by a factor of 10 every 5 years, data having a "primary home" in memory is now a reality. Main-memory databases eschew many of the traditional architectural pillars of relational database systems that optimized for disk-resident data. The result of these memory-optimized designs are systems that feature several innovative approaches to fundamental issues (e.g., concurrency control, query processing) that achieve orders of magnitude performance improvements over traditional designs. This monograph provides an overview of recent developments in main-memory database systems. It covers five main issues and architectural choices that need to be made when building a high performance main-memory optimized database: data organization and storage, indexing, concurrency control, durability and recovery techniques, and query processing and compilation. The monograph focuses on four commercial and research systems: H-Store/VoltDB, Hekaton, HyPer, and SAPHANA. These systems are diverse in their design choices and form a representative sample of the state of the art in main-memory database systems. It also covers other commercial and academic systems, along with current and future research trends.




Cache Conscious Column Organization in In-memory Column Stores


Book Description

Cost models are an essential part of database systems, as they are the basis of query performance optimization. Based on predictions made by cost models, the fastest query execution plan can be chosen and executed or algorithms can be tuned and optimised. In-memory databases shifts the focus from disk to main memory accesses and CPU costs, compared to disk based systems where input and output costs dominate the overall costs and other processing costs are often neglected. However, modelling memory accesses is fundamentally different and common models do not apply anymore. This work presents a detailed parameter evaluation for the plan operators scan with equality selection, scan with range selection, positional lookup and insert in in-memory column stores. Based on this evaluation, a cost model based on cache misses for estimating the runtime of the considered plan operators using different data structures is developed. Considered are uncompressed columns, bit compressed and dictionary encoded columns with sorted and unsorted dictionaries. Furthermore, tree indices on the columns and dictionaries are discussed. Finally, partitioned columns consisting of one partition with a sorted and one with an unsorted dictionary are investigated. New values are inserted in the unsorted dictionary partition and moved periodically by a merge process to the sorted partition. An efficient attribute merge algorithm is described, supporting the update performance required to run enterprise applications on read-optimised databases. Further, a memory traffic based cost model for the merge process is provided.




Database Machine Performance: Modeling Methodologies and Evaluation Strategies


Book Description

This book is focused on the performance evaluation of database machines, i.e., special-purpose architectures specifically meant to improve the efficiency of database applications. The topic is of primary interest because of the need to compare these systems among themselves and with traditional database management systems. The book gathers the experience of several European research groups in modeling and analyzing the database machine architectures they have proposed. It deals both with the main methodological issues and with the detailed analysis of some relevant problems. It also includes an extensive annotated bibliography with more than one hundred references and several keys for the access to the literature.