Skip to main content

Build System

This page describes the build and test framework the toolset ships under share/build/: flute.mk builds a program and test.mk runs its tests. A project finds them through the FLUTEMK and TESTMK variables.

Flute Model Compiler​

Pattern rules for using the flute model compiler are in flute.mk. A project Makefile includes them with include $(FLUTEMK) in its initial section. The rules run paint on bench.paint to make bench.obj and bench.map, link the image, and run the simulator on it.

bench.obj depends only on bench.paint. After editing a .w kernel, run make clean (or make -B) before testing again: an existing test directory is not rebuilt, and its old result is reported again.

Regression Tests​

Tests are organized as benches, one bench per directory. A bench is a narrowly constructed environment that exercises and validates some functionality. Most benches can be tested in multiple configurations. For example, the image convolution bench can be tested with different image sizes, and matrix sizes. A configuration file test.rc in each bench directory lists test configurations that are part of automated testing.

All benches have some common structure, that is required by the build system. The regression test environment is implemented in test.mk. Each bench is controlled by a Makefile that starts with the line include $(TESTMK). Each bench Makefile must provide the following targets:

  • test: The test recipe (and its dependants) perform compile-, run-, and verify- steps. The test settings are specified as environment variables. Test failure must be signalled by an error exit code from any of the steps. The test recipie should be adequately verbose for someone to re-create the test from the command line just by observing the output.

  • clean: The clean recipe removes all files created during execution of the test. After running clean, the directory holds only the bench's own files.

Specific tests are listed in test.rc, with one test configuration per line. A test can be de-activated by starting the line with a comment character #. Each test is a list of VARIABLE=VALUE bindings that will be set as environment variables when the test is run, and may also carry words without = that label it, such as tutorial or full.

From the command line, tests can be run as make TEST=<N> where N is the test's line number in test.rc, or as make TEST=<label>, which runs every line that carries the label. A list of tests can be run as make check TEST="1 2 5". All tests can be run as make check TEST=*. A failing test stops the run, and the tests after it are not run; make -k check TEST=* runs them all. If the test number is passed explicitly, it will be run even if it is de-activated. When all tests are selected with *, de-activated tests will be listed as SKIP in the test report. It is useful to put a brief comment on the skipped test line to explain why it has been de-activated. This comment will also appear in the test report.

A recursive suite can exclude whole benches from one named test selection without de-activating their configurations. Its top-level Makefile exports TEST_EXCLUDE_TAG, TEST_EXCLUDE_ROOT, and TEST_EXCLUDE_FILE before including test.mk. TEST_EXCLUDE_ROOT may be a common ancestor shared by multiple suite entry points. The exclusion file contains one bench directory per line, relative to that root; blank lines and # comments are ignored. Absolute paths, . or .. components, and paths without a Makefile are rejected. Exclusions apply only when that suite root recursively runs the named tag. Numeric selections and direct invocations inside a bench still run, which makes it possible to debug an excluded configuration explicitly.

Verbose make log output is present for each test in the file test-<N>/make-log, where <N> is the line's name if it carries one used nowhere else in test.rc, otherwise its line number.

To run a test configuration that is not present in test.rc, specify the test parameters directly on a test directory target: make test-1/test VARIABLE=VALUE... (the values override that line's settings)

To use the test configurations from test.rc with an alternative target, specify make TEST=<N> GOAL=foo

Note: test.mk provides a default check: target, that should not be extended or overridden.