Parallel Programming with Co-arrays


Book Description

Parallel Programming with Co-Arrays describes the basic techniques used to design parallel algorithms for high-performance, scientific computing. It is intended for upper-level undergraduate students and graduate students who need to develop parallel codes with little or no previous introduction to parallel computing. It is also intended as a reference manual for researchers active in the field of scientific computing. All the algorithms in the book are based on partition operators. These operators provide a unifying principle that fits seemingly disparate techniques into an overall framework for algorithm design. The book uses the co-array programming model to illustrate how to write code for concrete examples, but it emphasizes that the important concepts for algorithm design are independent of the programming model. With these concepts in mind, the reader can write algorithms in different programming models based on personal taste and comfort.




CoArrays


Book Description

This book describes the coarray parallel programming model that will be part of the next standard version of the Fortran language. It provides a practical guide for Fortran programmers who want to start writing parallel applications using coarrays as soon as the compilers become commercially available. The authors present the technical specification of the coarray model in enough detail for programmers to write standard-conforming code. They also offer a large number of examples of parallel algorithms written in the coarray model and include exercises with solutions. A supplementary website offers Fortran code samples for download.




Final Report, Center for Programming Models for Scalable Parallel Computing


Book Description

The major accomplishment of this project is the production of CafLib, an 'object-oriented' parallel numerical library written in Co-Array Fortran. CafLib contains distributed objects such as block vectors and block matrices along with procedures, attached to each object, that perform basic linear algebra operations such as matrix multiplication, matrix transpose and LU decomposition. It also contains constructors and destructors for each object that hide the details of data decomposition from the programmer, and it contains collective operations that allow the programmer to calculate global reductions, such as global sums, global minima and global maxima, as well as vector and matrix norms of several kinds. CafLib is designed to be extensible in such a way that programmers can define distributed grid and field objects, based on vector and matrix objects from the library, for finite difference algorithms to solve partial differential equations. A very important extra benefit that resulted from the project is the inclusion of the co-array programming model in the next Fortran standard called Fortran 2008. It is the first parallel programming model ever included as a standard part of the language. Co-arrays will be a supported feature in all Fortran compilers, and the portability provided by standardization will encourage a large number of programmers to adopt it for new parallel application development. The combination of object-oriented programming in Fortran 2003 with co-arrays in Fortran 2008 provides a very powerful programming model for high-performance scientific computing. Additional benefits from the project, beyond the original goal, include a programto provide access to the co-array model through access to the Cray compiler as a resource for teaching and research. Several academics, for the first time, included the co-array model as a topic in their courses on parallel computing. A separate collaborative project with LANL and PNNL showed how to extend the co-array model to other languages in a small experimental version of Co-array Python. Another collaborative project defined a Fortran 95 interface to ARMCI to encourage Fortran programmers to use the one-sided communication model in anticipation of their conversion to the co-array model later. A collaborative project with the Earth Sciences community at NASA Goddard and GFDL experimented with the co-array model within computational kernels related to their climate models, first using CafLib and then extending the co-array model to use design patterns. Future work will build on the design-pattern idea with a redesign of CafLib as a true object-oriented library using Fortran 2003 and as a parallel numerical library using Fortran 2008.




Modern Fortran


Book Description

Modern Fortran teaches you to develop fast, efficient parallel applications using twenty-first-century Fortran. In this guide, you’ll dive into Fortran by creating fun apps, including a tsunami simulator and a stock price analyzer. Filled with real-world use cases, insightful illustrations, and hands-on exercises, Modern Fortran helps you see this classic language in a whole new light. Summary Using Fortran, early and accurate forecasts for hurricanes and other major storms have saved thousands of lives. Better designs for ships, planes, and automobiles have made travel safer, more efficient, and less expensive than ever before. Using Fortran, low-level machine learning and deep learning libraries provide incredibly easy, fast, and insightful analysis of massive data. Fortran is an amazingly powerful and flexible programming language that forms the foundation of high performance computing for research, science, and industry. And it's come a long, long way since starting life on IBM mainframes in 1956. Modern Fortran is natively parallel, so it's uniquely suited for efficiently handling problems like complex simulations, long-range predictions, and ultra-precise designs. If you're working on tasks where speed, accuracy, and efficiency matter, it's time to discover—or re-discover—Fortran.. About the technology For over 60 years Fortran has been powering mission-critical scientific applications, and it isn't slowing down yet! Rock-solid reliability and new support for parallel programming make Fortran an essential language for next-generation high-performance computing. Simply put, the future is in parallel, and Fortran is already there. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the book Modern Fortran teaches you to develop fast, efficient parallel applications using twenty-first-century Fortran. In this guide, you'll dive into Fortran by creating fun apps, including a tsunami simulator and a stock price analyzer. Filled with real-world use cases, insightful illustrations, and hands-on exercises, Modern Fortran helps you see this classic language in a whole new light. What's inside Fortran's place in the modern world Working with variables, arrays, and functions Module development Parallelism with coarrays, teams, and events Interoperating Fortran with C About the reader For developers and computational scientists. No experience with Fortran required. About the author Milan Curcic is a meteorologist, oceanographer, and author of several general-purpose Fortran libraries and applications. Table of Contents PART 1 - GETTING STARTED WITH MODERN FORTRAN 1 Introducing Fortran 2 Getting started: Minimal working app PART 2 - CORE ELEMENTS OF FORTRAN 3 Writing reusable code with functions and subroutines 4 Organizing your Fortran code using modules 5 Analyzing time series data with arrays 6 Reading, writing, and formatting your data PART 3 - ADVANCED FORTRAN USE 7 Going parallel with Fortan coarrays 8 Working with abstract data using derived types 9 Generic procedures and operators for any data type 10 User-defined operators for derived types PART 4 - THE FINAL STRETCH 11 Interoperability with C: Exposing your app to the web 12 Advanced parallelism with teams, events, and collectives




Principles of Parallel Programming


Book Description

With the rise of multi-core architecture, parallel programming is an increasingly important topic for software engineers and computer system designers. Written by well-known researchers Larry Snyder and Calvin Lin, this highly anticipated first edition emphasises the principles underlying parallel computation, explains the various phenomena, and clarifies why these phenomena represent opportunities or barriers to successful parallel programming. Ideal for an advanced upper-level undergraduate course, Principles of Parallel Programming supplies enduring knowledge that will outlive the current hardware and software, aiming to inspire future researchers to build tomorrow's solutions.




Foundations of Parallel Programming


Book Description

This is the first comprehensive account of this new approach to the fundamentals of parallel programming.







Parallel Programming


Book Description

Mathematics of Computing -- Parallelism.




Parallel Programming


Book Description

Innovations in hardware architecture, like hyper-threading or multicore processors, mean that parallel computing resources are available for inexpensive desktop computers. In only a few years, many standard software products will be based on concepts of parallel programming implemented on such hardware, and the range of applications will be much broader than that of scientific computing, up to now the main application area for parallel computing. Rauber and Rünger take up these recent developments in processor architecture by giving detailed descriptions of parallel programming techniques that are necessary for developing efficient programs for multicore processors as well as for parallel cluster systems and supercomputers. Their book is structured in three main parts, covering all areas of parallel computing: the architecture of parallel systems, parallel programming models and environments, and the implementation of efficient application algorithms. The emphasis lies on parallel programming techniques needed for different architectures. For this second edition, all chapters have been carefully revised. The chapter on architecture of parallel systems has been updated considerably, with a greater emphasis on the architecture of multicore systems and adding new material on the latest developments in computer architecture. Lastly, a completely new chapter on general-purpose GPUs and the corresponding programming techniques has been added. The main goal of the book is to present parallel programming techniques that can be used in many situations for a broad range of application areas and which enable the reader to develop correct and efficient parallel programs. Many examples and exercises are provided to show how to apply the techniques. The book can be used as both a textbook for students and a reference book for professionals. The material presented has been used for courses in parallel programming at different universities for many years.




Parallel Processing and Applied Mathematics


Book Description

This volume comprises the proceedings of the 6th International Conference on Parallel Processing and Applied Mathematics - PPAM 2005, which was held in Poznan, the industrial, academic and cultural center in the western part of Poland, during September 11–14, 2005.