Writing & Running Tests

SuperJ has two test mechanisms. This guide covers both, but most SDK and application testing uses the first.

Rule of thumb: if you'd reach for assertEquals, write a unit test; if you'd println and eyeball the result, write a golden test.

Unit tests with sj.test

A first test

Create a file under test/ ending in Test.sj, mark it // UNIT, and write @Test static void methods. No main, no registration — the compiler wires it up:

// UNIT
import sj.util.IntArrayList;
import sj.test.Asserts;

class IntArrayListTest {
    @Test
    static void startsEmpty() {
        IntArrayList list = new IntArrayList();
        Asserts.assertTrue("new list is empty", list.isEmpty());
        Asserts.assertEquals("size", 0, list.size());
    }

    @Test
    static void addAndGet() {
        IntArrayList list = new IntArrayList();
        list.add(10);
        list.add(20);
        Asserts.assertEquals("size", 2, list.size());
        Asserts.assertEquals("first", 10, list.get(0));
    }
}

Run it:

superj test             # build compiler + SDK, then run every test/ suite

Output:

test IntArrayListTest.startsEmpty ... ok
test IntArrayListTest.addAndGet ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 filtered out

Rules

• File: under test/, with // UNIT in the first five lines (the marker superj test looks for). One class per file; name it <Thing>Test.sj.
• @Test marks a static void no-arg method. The compiler synthesizes the runner main that registers every @Test as "ClassName.methodName" and exits with the failure count. (Methods without @Test — helpers, capture() targets — are never run as tests.) It must be static because, with no reflection (SuperJ removed it), the runner can't instantiate a class or invoke an instance method at runtime — so registration is a compile-time method reference (ClassName::method, as in the explicit main below), and a no-arg method reference with no receiver has no instance to bind to.
• A test passes by returning and fails by throwing. An Asserts failure throws AssertionFailure; any other Throwable (e.g. an unexpected NPE) also counts as a failure, not a crash of the whole run.
• If you need a custom entry point you may still write an explicit main (build a TestRunner and call t.add("name", Class::method) by hand) — an explicit main suppresses the synthesized one. @Test is the common path.

Assertions — sj.test.Asserts

All are static, message-first, and throw AssertionFailure on failure (the message is printed under the FAILED line):

Testing that something throws

No reflection: capture() runs a body and hands you whatever it threw, then you check it with instanceof. The body is a method reference to a non-@Test helper.

class PoolTest {
    @Test
    static void rejectsNull() {
        Throwable t = Asserts.capture(PoolTest::offerNull);
        Asserts.assertTrue("throws IAE", t instanceof IllegalArgumentException);
    }

    static void offerNull() {           // helper — no @Test
        new ObjectPool<Box>(2, null).offer(null);
    }
}

Skipping a test — @Ignore

Add @Ignore (with @Test) to park a known-failing or flaky test. It is reported ... ignored, never run, and never counts as a failure:

@Test
@Ignore
static void flakyUntilFixed() {
    Asserts.assertEquals("known bad", 1, 2);   // skipped — suite stays green
}

test SomeTest.flakyUntilFixed ... ignored
test result: ok. 4 passed; 0 failed; 1 ignored; 0 filtered out

@Ignore without @Test is a compile error.

Reading SDK internals from a test — @VisibleForTesting

A test lives in a different package from the SDK class it exercises, so a private field or method of an SDK class is normally unreachable. Making it public pollutes the API; leaving it private makes the internal untestable.

@VisibleForTesting relaxes the private-access check only when the compiler is in test mode (i.e. superj test). Mark the member private and annotate it; a @Test in another package can then read/call it, but every non-test compile still enforces the private boundary:

// in an SDK class (sj.crypto.TlsEngineBase)
@VisibleForTesting
private static byte[] LABEL_BYTES_DERIVED_SECRET = encodeTls13Label("derived");

// in a test (default package) — test mode relaxes the private check
@Test
static void derivedSecretLabelAccessibleAcrossPackage() {
    byte[] label = TlsEngineBase.LABEL_BYTES_DERIVED_SECRET;
    Asserts.assertNotNull("field accessible via @VisibleForTesting", label);
    Asserts.assertEquals("label length", 13, label.length);
}

A regular superj compile (non-test) that tries the same access is still rejected with field '...' has private access in '...' — the relaxation only applies under superj test. The annotation works on private fields and private methods.

A/B differential tests

There's no special feature for this — compute a reference (oracle) result and the optimized/real result in one @Test, then assertEquals them over many inputs. Example: an insertion-sort oracle vs the SDK's Arrays.sort over seeded-random arrays (see test/SortDifferentialTest.sj):

@Test
static void arraysSortMatchesReference() {
    Random rng = new Random(12345L);
    for (int trial = 0; trial < 500; trial = trial + 1) {
        int n = rng.nextInt(64);
        int[] data = new int[n];
        for (int i = 0; i < n; i = i + 1) data[i] = rng.nextInt(1000) - 500;
        int[] expected = referenceSort(data);          // A: trivially-correct oracle
        int[] actual = new int[n];                      // B: the path under test
        System.arraycopy(data, 0, actual, 0, n);
        Arrays.sort(actual);
        for (int i = 0; i < n; i = i + 1)
            Asserts.assertEquals("trial " + trial + " idx " + i, expected[i], actual[i]);
    }
}

Running tests

superj test                          # the standard gate: build + run all of test/
superj test                          # same, if compiler + SDK are already built (dir defaults to test/)
superj test test --filter pool       # only cases whose name contains "pool"

superj test:

• discovers every // UNIT file under the directory (default test/),
• compiles each suite against the SDK (archive --sdk-path fast path first, falling back to whole-program --sdk-source if no archive is present) and links it,
• runs it, and aggregates pass/fail by exit code — a non-zero suite makes superj test exit non-zero (CI-ready).

Options: --filter <substr> (forwarded to each suite; runs only matching cases), --sdk-path <dir> (default $SJ_HOME/sdk), --sdk-source <dir> (default $SJ_HOME/sdk/sj — VIP install only), --clang-path <path>.

Project mode: caching, suite selection, parallel builds

Inside a project/workspace (superj test / superj test -p <member>), the runner additionally:

• Caches suite binaries. Each suite binary under target/test/ is stamped with the hash of [library sources + SDK variant + compiler version + flags + the suite file itself] — the same machinery as superj build's up-to-date check. An unchanged suite skips compile+link entirely (its tests still run); editing one test file rebuilds exactly one binary, editing a src/ helper rebuilds all. Each result line shows provenance and timing: PASS test/MemPoolTest.sj (312ms, cached). superj clean clears the cache.
• --suite <substr> selects suites by file path before the build loop — the single-suite edit-compile-run cycle no longer pays for the other suites. (--filter remains the runtime case filter within a suite.)
• --jobs <n> builds out-of-date suites in parallel (default: online cores, capped by the number of suites to build; the links are independent and dominate wall time). Compile failures point at a per-suite target/test/<suite>.buildlog.
• --debug is forwarded to each suite compile (DWARF line info, -O0 link). When a suite binary dies on a signal (exit code > 128, e.g. a segfault from a null deref or stack overflow), the runner automatically re-runs it under lldb (macOS) / gdb (Linux) and prints the faulting .sj source line, so a crashing test reports file:line instead of a bare "Segmentation fault". Crashes inside native C leaf functions (which lack frame pointers) print the signal/stop reason but may not resolve a .sj frame.

The summary line reports cache effectiveness: test: 45 passed, 0 failed (1 built, 44 cached, jobs=10).

Under an enterprise SDK (precompiled module, no archive), suite builds fast-link against a cached native compile of the SDK instead of re-optimizing the merged whole-program IR per suite — a changed suite rebuilds in ~0.3 s instead of ~8 s. The prelink object is built once (serially, before any --jobs fan-out) and cached beside the SDK module.

Gotchas

• Unit tests are not run by the golden suite — they're a separate step. A change to SDK logic should run both superj golden and superj test.
• Rebuild the SDK after a compiler change before running either suite, or stale-archive link errors look like test failures.
• Suites compile against the SDK (archive fast path, --sdk-source fallback), so cross-class static final int constants read from another class can surface as 0 in some situations; prefer a method or a same-class constant if you hit it.

Golden-output tests

For conformance and compiler behavior, where the printed output is the assertion. Put a program in examples/<name>.sj and its expected stdout in examples/expected_<name>.txt; superj golden compiles, runs, and diffs.

// examples/test_my_feature.sj
public class test_my_feature {
    public static void main(String[] args) {
        System.out.println("answer=" + (6 * 7));
    }
}

# examples/expected_test_my_feature.txt
answer=42

superj golden           # runs all categories: positive, negative,
                        # smoke, SDK positive/smoke/compile

Other golden conventions (first five lines of the file): // ERROR: <substr> marks a negative test that must fail to compile with that message; // WARNING: <substr> a compile that must warn. Don't put unit-logic assertions in a golden test — every new check forces regenerating the golden file, and the first mismatch hides the rest. Use a sj.test unit test instead.