Collective Communication
These topics combine values from many PEs, or give one value to many, in four ways:
- in the routers, with the fabric adder;
- with switches that restore themselves after each round;
- with a barrier and a broadcast built from switch positions;
- with messages that carry their own addresses, on a tree placed on any PEs.
Each topic is a complete program, and each opens with the lessons and topics it needs. Read a topic once you have read everything on its Needs line.
| Topic | What it teaches |
|---|---|
| Reductions in the router: the fabric adder | The adder on color 10, pairing by arrival order, its number formats and limits, and a streaming convolution |
Switch protocols: remote advance, pop, rings, self-restoring barriers | Control words with several commands, pop and ring, and a row collective that leaves its switches ready for the next round |
| A barrier and a broadcast from switch positions | Closed positions that a PE opens itself, a probe whose return releases the barrier, a broadcast and a reset on the same color |
| Collectives over messages: tree reduction and prefix scan | A logical tree placed on any PEs, arrival in no fixed order, sender tags, and a two-phase scan |