Lesson 7 — Tasks: code that runs when a wavelet arrives

The kernels so far receive a fixed number of values in a fixed order. Some kernels must react to what arrives instead: several kinds of message on one color, or a stream whose length the receiver does not know. A dispatch runs a piece of code, a handler, each time a message arrives.

What you will be able to do

• Write a dispatch with several handlers, and send the messages that call them.
• Keep state on a PE from one message to the next, and reset it with a message.
• Say what the hardware runs: each handler is a task that an arriving wavelet starts.
• Write a handler that ignores the contents of its wavelet.

The program

compute holds a 4×4 matrix A. send sends it a vector x, one element per message, then a flush() message. For each element, compute adds A[i][j] * x[i] to Y[j] for every j. On flush() it sends Y to recv and resets Y to zero.

Y[j] is the sum over i of A[i][j] * x[i]: column j of A times x. So Y is Aᵀx, the transpose of A times x.

          rxcolor               txcolor
 +------+          +---------+          +------+
 | send | -------> | compute | -------> | recv |
 +------+          +---------+          +------+
  [C>R]          rxcolor [L>C]           [L>C]
                 txcolor [C>R]

NVEC is the number of vectors. The tutorial row sends one, x = (1, 2, 3, 4), and recv.y ends as {80, 90, 100, 110}. The two row sends a second vector, x = (4, 3, 2, 1), after the first; its result is {40, 50, 60, 70}. The second result shows that flush() reset Y.

The code

compute.w: the dispatch

xp param N, W;

socket rx;
socket tx;

hp Y[m=W] = {0.0};

hp A[n=N][m=W] = {
    {  0.,  1.,  2.,  3. },
    {  4.,  5.,  6.,  7. },
    {  8.,  9., 10., 11. },
    { 12., 13., 14., 15. },
};

let j in [0, W);

forever matvec dispatch(rx) {
    wavelet(hp x, xp i) {
        ∀j Y[j] <+- A[i][j] * x;
    }
    flush() {
        ∀j tx[] <- Y[j];
        ∀j Y[j] <- 0.;
    }
}

Part	Meaning
dispatch(rx) { ... }	Wait for messages on socket rx, and run the handler that each message names.
forever	Re-arm the dispatch after each handler, so that it handles every message, not only the first.
matvec	An optional name. The listing calls the handlers' tasks matvec_wavelet and matvec_flush.
wavelet(hp x, xp i) { ... }	The handler for data wavelets. wavelet is a reserved name: plain data wavelets start this handler, and a call tx[] <- wavelet(...) sends data wavelets, not a control wavelet. Its arguments are the values the sender passed.
flush() { ... }	A named message. Any name other than wavelet works if the sender's call uses it: eov() would work as well as flush(). The call sends a control wavelet that names the handler's task. Plain sends never start a named handler.
Y[j] <+- A[i][j] * x	Accumulate: add A[i][j] * x to Y[j]. The compiler fuses the multiply and the add, so each step rounds once; ref.py computes the same way.

hp A[n=N][m=W] is a 2-D array with dimensions n and m. Its initializer lists the rows, and memory holds the array row by row, so A[i][j] is element W*i + j. xp param N, W; declares two parameters in one statement.

Y is an ordinary array, so it keeps its value from one handler to the next. That is the state of this kernel. flush() sends it and resets it for the next vector.

send.w: the messages

xp param N;
xp param NVEC;

socket tx;

hp x[v=NVEC][n=N];

let v in [0, NVEC);
let i in [0, N);
∀v {
    ∀i tx[] <- wavelet(x[v][i], i);
    tx[] <- flush();
}

∀v { ... } runs the block once for each vector: its N elements, then a flush(). tx[] <- wavelet(x[v][i], i); sends a message that calls the handler wavelet of the kernel at the other end of tx, with the arguments x[v][i] and i. tx[] <- flush(); calls flush. Wavelets on one color stay in the order they were sent, so each flush() arrives after its vector's elements.

ref.py writes the vectors to x.bin, and the Makefile loads them into send.x with miset ... order="v n": the file holds x[0][0..N), then x[1][0..N). recv.w receives NVEC results of W values each with ∀v ∀j y[v][j] <- rx[];, which runs one statement over both ranges, j inside v.

How the sender knows the handlers

The sender's code depends on the receiver's handlers, so Paint reads the handlers of every kernel first and then compiles each kernel against the handlers at the other end of its sockets. In test-tutorial/make-log, each kernel's carbide command appears twice: first with an rpc= term, which collects the handlers, then with rpco=, which compiles against them.

A call that no handler matches fails the build. With done() in place of flush() in send.w:

../send.w: error: RPC function declaration for [done] not found in scope.

What the hardware does

Each handler is a task

The listing of compute.w (in test-tutorial/compute_artefacts/) begins:

task main id(8) status(active) {
	ublk 0x0000;  //@../compute.w:17
}
task matvec_wavelet id(0) status(block) {
	fmach [T23] = [T01], [T12], R2;  //@../compute.w:19
}

• The kernel's top level, task 8, unblocks task 0 and ends. Then the core has nothing to run until a wavelet arrives.
• A data wavelet that reaches rx's input queue starts that queue's task, task 0, which is the wavelet handler. When a task starts, the core copies its wavelet into registers: R2 gets the lower 16 bits, which carry x, and R3 and R4 get the upper 16 bits, the index.
• The whole handler is one vector instruction, a multiply-add over the four elements of Y. Its operand T12 reads four elements of A. The hardware adds the wavelet's upper 16 bits, its index, to that operand's start address, so the instruction reads row i.
• flush() is a control wavelet: its control bit is set. The low six bits of its index name the task it starts, task 63, matvec_flush. That task blocks task 0 while it runs, sends Y, resets it, and unblocks task 0 again.

Each task runs until it ends; then the core picks the next ready task (lesson 5).

What the handler receives

The handler gets the values the sender passed, and the compiler chooses how to encode them. Here i serves only as an index into A, so the sender puts i times the row length, 4 here, in the upper half of the wavelet: the index is added to an address, and row i starts 4*i elements after row 0. In the tutorial row's listing of send.w, sll16 R2 = R0, 0x0002 multiplies i by 4, and merge32 [T22] = R2, R1 sends it with x in the lower half: one wavelet per call. The register numbers can differ between rows and builds. You never write the factor yourself. A test handler that also added i to a counter, for i = 0, 1, 2, 3, counted 6, the values the sender passed; the sender then sent two wavelets per call.

A handler that waits

flush()'s send runs on the main thread: the listing declares it with .upr=NONE. If recv stopped reading, the send would wait for room, the main thread would be suspended, and no other task would start (lesson 6). The next vector's messages would wait in compute's input queue, then in the fabric, and then send's sends would wait too.

Handlers that ignore their wavelet

A handler with no arguments, wavelet() { ... }, is started by a data wavelet and ignores what that wavelet carries. It must be named wavelet: a handler with any other name waits for a control wavelet with its task ID, so plain sends never start it. A handler can also take an argument and not use it, as in wavelet(hp w) { }. Lessons 8 and 11 put such kernels at the edges of a region: one absorbs what the last PE sends, and one answers each wavelet with a 0 (a[] <- 0:sp;).

Such a handler runs once per full wavelet, not once per value. A PE sent five xp values with plain sends, ∀i tx[] <- v[i];, to a wavelet() handler that counted its calls. The handler ran twice: the five values travel as two full wavelets and a half-wavelet, and the half-wavelet did not start it. A wavelet(xp v) handler that summed v got 4, which is 1 + 3, the first value of each pair. Send to a dispatch with calls such as wavelet(...), or with 32-bit values.

When to use a dispatch

• Several kinds of message share one color, as the elements and flush() do here.
• The receiver does not know how many values will come: a named message such as flush() ends the stream.
• A PE must answer whatever arrives, as the edge kernels of lessons 8 and 11 do.

A stream of known length in a known order is simpler as a receive loop (lesson 4). When a named message ends a stream, give it a row in the stream's ledger (lesson 6), and check that the sender sends it.

Run it

In your copy of the tutorial (see Lesson 1):

cd tutorial/main-2-between-pes/07-tasks-and-dispatch
make TEST=tutorial
make TEST=two

A pass prints a line ending in PASS tutorial [ N=4 W=4 NVEC=1 ARCH=sdr ], then one for two. ref.py reads A from the image, computes each result with one rounding per multiply-add, and the Makefile compares it with recv.y.

Pitfalls

• A handler for plain sends must be named wavelet. A dispatch whose only handler is edge() { ... } compiles, but plain sends never run it, and nothing reports this.
• A call that no handler matches fails the build with RPC function declaration for [...] not found in scope.
• Two receivers on one color with different handlers of one name. Paint compiles the sender against all of them. When one PE on a color had wavelet() and another had wavelet(xp v), the build failed with Cannot define functions with the same name but different signatures on the same socket.
• State that is not reset. A handler that ends a stream must reset what the next stream accumulates into.
• Plain 16-bit sends to a dispatch. The handler runs once per wavelet, not once per value.
• A send in a handler waits like any send on the main thread, and while it waits no other handler runs.

Exercise

Remove ∀j Y[j] <- 0.; from flush(). Predict the two row's second result, then run make clean and make TEST=two. The row fails. Read recv.y as lesson 4's Run it shows, with test-two and TEST=two in place of test-tutorial and TEST=tutorial, and hp in place of decimal. Why does the first result stay right?

Next

Lesson 8 — Space and time: a wavefront along a row

References

• Dispatch blocks
• Handler completion and forever
• Sending structured wavelets
• Accumulate assignment (<+-)
• Architecture notes: Tasks, Wavelets