Inside the Message Passing Interface

- Introduction - Learn what expects you inside the book
- What this book is about
- Who should read this book
- Notation and conventions
- How to read this book
- Overview
- Parallel computer
- Intraprocessor parallelism
- Interprocessor parallelism
- Exercises
- MPI standard
- MPI history
- Related standards
- Exercises
- MPI subsetting
- Motivation
- Typical examples
- Implementation practice
- Exercises
- Shared memory - Learn how to create a simple MPI subset capable of basic blocking point-to-point and collective operations over shared memory
- Subset definition
- General assumptions
- Blocking point-to-point communication
- Blocking collective operations
- Exercises
- Communication mechanisms
- Basic communication
- Intraprocess performance
- Interprocess performance
- Exercises
- Startup and termination
- Process creation
- Two processes
- More processes
- Connection establishment
- Process termination
- Exercises
- Blocking point-to-point communication
- Limited message length
- Blocking protocol
- Unlimited message length
- Double buffering
- Eager protocol
- Rendezvous protocol
- Exercises
- Blocking collective operations
- Naive algorithms
- Barrier
- Broadcast
- Reduce and Allreduce
- Exercises
- Sockets - Learn how to create an MPI subset capable of all point-to-point and blocking collective operations over Ethernet and other IP capable networks
- Subset definition
- General assumptions
- Blocking point-to-point communication
- Nonblocking point-to-point operations
- Blocking collective operations
- Exercises
- Communication mechanisms
- Basic communication
- Intranode performance
- Internode performance
- Exercises
- Synchronous progress engine
- Communication establishment
- Data transfer
- Exercises
- Startup and termination
- Process creation
- Startup command
- Process daemon
- Out-of-band communication
- Host name resolution
- Connection establishment
- At startup (eager)
- On request (lazy)
- Process termination
- Exercises
- Blocking point-to-point communication
- Source and tag matching
- Unexpected messages
- Exercises
- Nonblocking point-to-point communication
- Request management
- Exercises
- Blocking collective operations
- Communication context
- Basic algorithms
- Tree based algorithms
- Circular algorithms
- Hypercube algorithms
- Exercises
- OFA libfabrics - Learn how to create an MPI subset capable of all point-to-point and collective operations over InfiniBand and upcoming future networks
- Subset definition
- General assumptions
- Point-to-point operations
- Collective operations
- Exercises
- Communication mechanisms
- Basic communication
- Intranode performance
- Internode performance
- Exercises
- Startup and termination
- Process creation
- Credential exchange
- Connection establishment
- Process termination
- Exercises
- Point-to-point communication
- Blocking communication
- Nonblocking communication
- Exercises
- Collective operations
- Advanced algorithms
- Blocking operations
- Nonblocking operations
- Exercises
- Advanced features - Learn how to add advanced MPI features including but not limited to heterogeneity, one-sided communication, file I/O, and language bindings
- Communication modes
- Standard
- Buffered
- Synchronous
- Heterogeneity
- Basic datatypes
- Simple datatypes
- Derived datatypes
- Exercises
- Groups, communicators, topologies
- Group management
- Communicator management
- Process topologies
- Exercises
- One-sided communication
- Mapped implementation
- Native implementation
- Exercises
- File I/O
- Standard I/O
- MPI file I/O
- Exercises
- Language bindings
- Fortran
- C++
- Java
- Python
- Exercises
- Optimization - Learn how to optimize MPI internally by using advanced implementation techniques and available special hardware
- Direct data transfer
- Direct memory access
- Remote direct memory access
- Exercises
- Threads
- Thread support level
- Threads as MPI processes
- Shared memory extensions
- Exercises
- Multiple fabrics
- Synchronous progress engine
- Asynchronous progress engine
- Hybrid progress engine
- Exercises
- Dedicated hardware
- Synchronization
- Special memory
- Auxiliary networks
- Exercises
- Look ahead - Learn to recognize MPI advantages and drawbacks to better assess its future
- MPI axioms
- Reliable data transfer
- Ordered message delivery
- Dense process rank sequence
- Exercises
- MPI-4 en route
- Fault tolerance
- Exercises
- Beyond MPI
- Exascale challenge
- Exercises
- References - Learn about books that may further extend your knowledge
Appendices - MPI Families - Learn about major MPI implementation families, their genesis, architecture and relative performance
- MPICH
- Genesis
- Architecture
- Details
- MPICH
- MVAPICH
- Intel MPI
- .
- Exercises
- OpenMPI
- Genesis
- Architecture
- Details
- Exercises
- Comparison
- Market
- Features
- Performance
- Exercises
- Alternative interfaces - Learn about other popular interfaces that are used to implement MPI
- DAPL
- .
- Exercises
- SHMEM
- .
- Exercises
- GasNET
- .
- Exercises
- Portals
- .
- Exercises
- Solutions to all exercises - Learn how to answer all those questions