Topics in Parallel and Distributed Computing

This book introduces beginning undergraduate students of computing and computational disciplines to modern parallel and distributed programming languages and environments, including map-reduce, general-purpose graphics processing units (GPUs), and graphical user interfaces (GUI) for mobile applications. The book also guides instructors via selected essays on what and how to introduce parallel and distributed computing topics into the undergraduate curricula, including quality criteria for parallel algorithms and programs, scalability, parallel performance, fault tolerance, and energy efficiency analysis. The chapters designed for students serve as supplemental textual material for early computing core courses, which students can use for learning and exercises. The illustrations, examples, and sequences of smaller steps to build larger concepts are also tools that could be inserted into existing instructor material. The chapters intended for instructors are written at a teaching level and serve as a rigorous reference to include learning goals, advice on presentation and use of the material, within early and advanced undergraduate courses. 

Since Parallel and Distributed Computing (PDC) now permeates most computing activities, imparting a broad-based skill set in PDC technology at various levels in the undergraduate educational fabric woven by Computer Science (CS) and Computer Engineering (CE) programs as well as related computational disciplines has become essential. This book and others in this series aim to address the need for lack of suitable textbook support for integrating PDC-related topics into undergraduate courses, especially in the early curriculum. The chapters are aligned with the curricular guidelines promulgated by the NSF/IEEE-TCPP Curriculum Initiative on Parallel and Distributed Computing for CS and CE students and with the CS2013 ACM/IEEE Computer Science Curricula.

Anshul Gupta is a Principal Research Staff Member in IBM Research AI at IBM T.J. Watson Research Center. His research interests include sparse matrix computations and their applications in optimization and computational sci- ences, parallel algorithms, and graph/combinatorial algo- rithms for scientific computing. He has coauthored several journal articles and conference papers on these topics and a textbook titled ”Introduction to Parallel Computing.” He is the primary author of Watson Sparse Matrix Package (WSMP), one of the most robust and scalable parallel direct solvers for large sparse systems of linear equations.

Sushil K. Prasad (BTech’85 IIT Kharagpur, MS’86 Washington State, Pullman; PhD’90 Central Florida, Orlando - all in Computer Science/Engineering) is a Professor of Computer Science at Georgia State University and Director of Distributed and Mobile Systems (DiMoS) Lab. Sushil has been honored as an ACM Distinguished Scientist in Fall 2013 for his research on parallel data structures and applications. He was the elected chair of IEEE Technical Committee on Parallel Processing for two terms (2007-11), and received its highest honors in 2012 - IEEE TCPP Outstanding Service Award. Currently, he is leading the NSF-supported IEEE- TCPP curriculum initiative on parallel and distributed computing with a vision to  ensure  that  all  computer  science and engineering graduates are well-prepared in parallelism through their core courses in this era of multi- and many- cores desktops and handhelds. His current research interests are in Parallel Data Structures and Algorithms, and Computation over Geo-Spatiotemporal Datasets over Cloud, GPU  and  Multicore Platforms. Sushil is  currently a  Program Director leading the Office of Advanced Cyberinfrastructure (OAC) Learning and Workforce Development crosscutting programs at U.S. National Science Foundation. H

Arnold L. Rosenberg holds the rank of Distinguished University Professor Emeritus in the School of Computer Science at   the University of Massachusetts Amherst. Prior to joining UMass, Rosenberg was a Professor of Computer Science at Duke University from 1981 to 1986, and a Research Staff Member at the IBM Watson Research Center from 1965 to 1981. He has held visiting positions at Yale University and the University of Toronto, as well as research professorships at Colorado State University  and  Northeastern  University. He was a Lady Davis Visiting Professor at the Technion (Israel Institute of Technology) in 1994, and a Fulbright Senior Research Scholar at the University of Paris-South in 2000. Rosenberg’s research focuses on developing algorithmic models and techniques to exploit the new modalities of “collaborative computing” (wherein multiple computers cooperate to solve a computational problem)  that  result  from emerging computing technologies. Rosenberg is the author or coauthor of more than 190 technical papers on  these and other topics in theoretical computer science and discrete mathematics.  He is the coauthor of the research  book Graph Separators, with Applications and the author of the textbook The Pillars of Computation Theory: State, Encoding, Nondeterminism; additionally, he has served as coeditor of several books. Rosenberg is a Life Fellow of the ACM, a Life Fellow of the IEEE, a Golden Core member of the IEEE Computer Society, and a member of the Sigma Xi Research Society. Rosenberg received an A.B. in mathematics at Harvard College and an A.M. and Ph.D. in applied mathematics  at  Harvard  University.

Alan Sussman is a Professor in the Department of Computer Science and Institute for Advanced  Computer  Studies  at  the University of Maryland. Working with  students  and other researchers at Maryland and other institutions he has published numerous conference and journal papers and re- ceived several best paper awards in various topics related to software tools for high performance parallel and distributed computing, and has contributed chapters to 6 books. His research interests include peer-to-peer distributed systems, software engineering for high performance computing, and large scale data  intensive  computing.  Software  tools  he has built with his graduate students have been widely dis- tributed and used in many computational science applica- tions, in areas such as earth science, space science, and med- ical informatics. He is a subject area editor for the Parallel Computing journal and an associate editor for IEEE Trans- actions on Services Computing, and edited a previous book on teaching parallel and distributed computing. He is a founding member of the Center for Parallel and Distributed Computing Curriculum Development and Educational Re- sources (CDER). He received his Ph.D. in computer science from Carnegie Mellon  University.

Charles Weems is co-director of the Architecture and Language Implementation lab at the University of Massachusetts. His current research interests include architectures for media and embedded applications, GPU computing, and high precision arithmetic, and he has over 100 conference and journal publications. Previously he led development of two generations of a heterogeneous parallel processor for machine vision, called the Image Understanding Architecture, and co-directed initial work on the Scale compiler that was eventually used for the TRIPS architecture. He is the author of numerous articles, has served on many program committees, chaired the 1997 IEEE CAMP Workshop, the 1999 IEEE Frontiers Symposium, co-chaired IEEE IPDPS in 1999, 2000, and 2013, was general vice-chair for IPDPS from 2001 through 2005, is on the steering committees of EduPar and EduHPC. He has co-authored twenty-eight introductory CS texts. He  is a member of ACM, Senior Member of IEEE, a member of the Executive Committee of the IEEE TC on Parallel Processing, has been an editor for IEEE TPDS, Elsevier JPDC, and is an editor with Parallel Computing.