Scalable Parallel Computing


Book Description

This book covers four areas of parallel computing: principles, technology, architecture, and programming. It is suitable for professionals and undergraduates taking courses in computer engineering, parallel processing, computer architecture, scaleable computers or distributed computing.




Scaling Up Machine Learning


Book Description

This integrated collection covers a range of parallelization platforms, concurrent programming frameworks and machine learning settings, with case studies.




Scalable Parallel Programming Applied to H.264/AVC Decoding


Book Description

Existing software applications should be redesigned if programmers want to benefit from the performance offered by multi- and many-core architectures. Performance scalability now depends on the possibility of finding and exploiting enough Thread-Level Parallelism (TLP) in applications for using the increasing numbers of cores on a chip. Video decoding is an example of an application domain with increasing computational requirements every new generation. This is due, on the one hand, to the trend towards high quality video systems (high definition and frame rate, 3D displays, etc) that results in a continuous increase in the amount of data that has to be processed in real-time. On the other hand, there is the requirement to maintain high compression efficiency which is only possible with video codes like H.264/AVC that use advanced coding techniques. In this book, the parallelization of H.264/AVC decoding is presented as a case study of parallel programming. H.264/AVC decoding is an example of a complex application with many levels of dependencies, different kernels, and irregular data structures. The book presents a detailed methodology for parallelization of this type of applications. It begins with a description of the algorithm, an analysis of the data dependencies and an evaluation of the different parallelization strategies. Then the design and implementation of a novel parallelization approach is presented that is scalable to many core architectures. Experimental results on different parallel architectures are discussed in detail. Finally, an outlook is given on parallelization opportunities in the upcoming HEVC standard.




Structured Parallel Programming


Book Description

Programming is now parallel programming. Much as structured programming revolutionized traditional serial programming decades ago, a new kind of structured programming, based on patterns, is relevant to parallel programming today. Parallel computing experts and industry insiders Michael McCool, Arch Robison, and James Reinders describe how to design and implement maintainable and efficient parallel algorithms using a pattern-based approach. They present both theory and practice, and give detailed concrete examples using multiple programming models. Examples are primarily given using two of the most popular and cutting edge programming models for parallel programming: Threading Building Blocks, and Cilk Plus. These architecture-independent models enable easy integration into existing applications, preserve investments in existing code, and speed the development of parallel applications. Examples from realistic contexts illustrate patterns and themes in parallel algorithm design that are widely applicable regardless of implementation technology. The patterns-based approach offers structure and insight that developers can apply to a variety of parallel programming models Develops a composable, structured, scalable, and machine-independent approach to parallel computing Includes detailed examples in both Cilk Plus and the latest Threading Building Blocks, which support a wide variety of computers




Compiler Optimizations for Scalable Parallel Systems


Book Description

Scalable parallel systems or, more generally, distributed memory systems offer a challenging model of computing and pose fascinating problems regarding compiler optimization, ranging from language design to run time systems. Research in this area is foundational to many challenges from memory hierarchy optimizations to communication optimization. This unique, handbook-like monograph assesses the state of the art in the area in a systematic and comprehensive way. The 21 coherent chapters by leading researchers provide complete and competent coverage of all relevant aspects of compiler optimization for scalable parallel systems. The book is divided into five parts on languages, analysis, communication optimizations, code generation, and run time systems. This book will serve as a landmark source for education, information, and reference to students, practitioners, professionals, and researchers interested in updating their knowledge about or active in parallel computing.




Annual Review of Scalable Computing


Book Description

This book contains four review articles in the area of scalable computing. Two of the articles discuss methods and tools for the parallel solution of irregular problems, which have been satisfactorily worked out in heterogeneous systems. One surveys the technology and applications of multimedia server clusters, which are playing an increasing role in the current networked environment. An additional article discusses SilkRoad, which adds distributed shared memory capabilities to the Cilk parallel programming system. Once again, the book represents a new set of steps forward in parallel systems. Graduate students, academics and researchers in supercomputing and computer engineering.




Annual Review of Scalable Computing


Book Description

The third volume in the Series on Scalable Computing, this book contains five new articles describing significant developments in the field. Included are such current topics as clusters, parallel tools, load balancing, mobile systems, and architecture independence.




Annual Review of Scalable Computing


Book Description

Comprehensively discusses significant projects in scalable computing in various research organizations around the world.







Languages, Compilers, and Run-Time Systems for Scalable Computers


Book Description

This book constitutes the strictly refereed post-workshop proceedings of the 4th International Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computing, LCR '98, held in Pittsburgh, PA, USA in May 1998. The 23 revised full papers presented were carefully selected from a total of 47 submissions; also included are nine refereed short papers. All current issues of developing software systems for parallel and distributed computers are covered, in particular irregular applications, automatic parallelization, run-time parallelization, load balancing, message-passing systems, parallelizing compilers, shared memory systems, client server applications, etc.