Writing & Running Tests

SuperJ has two test mechanisms. This guide covers both, but most SDK and application testing uses the first.

Rule of thumb: if you'd reach for assertEquals, write a unit test; if you'd println and eyeball the result, write a golden test.

Unit tests with sj.test

A first test

Create a file under test/ ending in Test.sj, mark it // UNIT, and write @Test static void methods. No main, no registration — the compiler wires it up:

// UNIT
import sj.util.IntArrayList;
import sj.test.Asserts;

class IntArrayListTest {
    @Test
    static void startsEmpty() {
        IntArrayList list = new IntArrayList();
        Asserts.assertTrue("new list is empty", list.isEmpty());
        Asserts.assertEquals("size", 0, list.size());
    }

    @Test
    static void addAndGet() {
        IntArrayList list = new IntArrayList();
        list.add(10);
        list.add(20);
        Asserts.assertEquals("size", 2, list.size());
        Asserts.assertEquals("first", 10, list.get(0));
    }
}

Run it:

make test-unit          # build compiler + SDK, then run every test/ suite

Output:

test IntArrayListTest.startsEmpty ... ok
test IntArrayListTest.addAndGet ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 filtered out

Rules

• File: under test/, with // UNIT in the first five lines (the marker superj test looks for). One class per file; name it <Thing>Test.sj.
• @Test marks a static void no-arg method. The compiler synthesizes the runner main that registers every @Test as "ClassName.methodName" and exits with the failure count. (Methods without @Test — helpers, capture() targets — are never run as tests.) It must be static because, with no reflection (SuperJ removed it), the runner can't instantiate a class or invoke an instance method at runtime — so registration is a compile-time method reference (ClassName::method, as in the explicit main below), and a no-arg method reference with no receiver has no instance to bind to.
• A test passes by returning and fails by throwing. An Asserts failure throws AssertionFailure; any other Throwable (e.g. an unexpected NPE) also counts as a failure, not a crash of the whole run.
• If you need a custom entry point you may still write an explicit main (build a TestRunner and call t.add("name", Class::method) by hand) — an explicit main suppresses the synthesized one. @Test is the common path.

Assertions — sj.test.Asserts

All are static, message-first, and throw AssertionFailure on failure (the message is printed under the FAILED line):

Testing that something throws

No reflection: capture() runs a body and hands you whatever it threw, then you check it with instanceof. The body is a method reference to a non-@Test helper.

class PoolTest {
    @Test
    static void rejectsNull() {
        Throwable t = Asserts.capture(PoolTest::offerNull);
        Asserts.assertTrue("throws IAE", t instanceof IllegalArgumentException);
    }

    static void offerNull() {           // helper — no @Test
        new ObjectPool<Box>(2, null).offer(null);
    }
}

Skipping a test — @Ignore

Add @Ignore (with @Test) to park a known-failing or flaky test. It is reported ... ignored, never run, and never counts as a failure:

@Test
@Ignore
static void flakyUntilFixed() {
    Asserts.assertEquals("known bad", 1, 2);   // skipped — suite stays green
}

test SomeTest.flakyUntilFixed ... ignored
test result: ok. 4 passed; 0 failed; 1 ignored; 0 filtered out

@Ignore without @Test is a compile error.

A/B differential tests

There's no special feature for this — compute a reference (oracle) result and the optimized/real result in one @Test, then assertEquals them over many inputs. Example: an insertion-sort oracle vs the SDK's Arrays.sort over seeded-random arrays (see test/SortDifferentialTest.sj):

@Test
static void arraysSortMatchesReference() {
    Random rng = new Random(12345L);
    for (int trial = 0; trial < 500; trial = trial + 1) {
        int n = rng.nextInt(64);
        int[] data = new int[n];
        for (int i = 0; i < n; i = i + 1) data[i] = rng.nextInt(1000) - 500;
        int[] expected = referenceSort(data);          // A: trivially-correct oracle
        int[] actual = new int[n];                      // B: the path under test
        System.arraycopy(data, 0, actual, 0, n);
        Arrays.sort(actual);
        for (int i = 0; i < n; i = i + 1)
            Asserts.assertEquals("trial " + trial + " idx " + i, expected[i], actual[i]);
    }
}

Running tests

make test-unit                          # the standard gate: build + run all of test/
./tools/superj test                     # same, if compiler + SDK are already built (dir defaults to test/)
./tools/superj test test --filter pool  # only cases whose name contains "pool"

superj test:

• discovers every // UNIT file under the directory (default test/),
• compiles each --sdk-source and links it,
• runs it, and aggregates pass/fail by exit code — a non-zero suite makes superj test exit non-zero (CI-ready).

Options: --filter <substr> (forwarded to each suite; runs only matching cases), --sdk-source <dir> (default sdk/sj), --clang-path <path>.

Gotchas

• Unit tests are not run by the golden suite — they're a separate step. A change to SDK logic should run both make test-golden and make test-unit.
• Rebuild the SDK after a compiler change (make sdk) before running either suite, or stale-archive link errors look like test failures.
• Suites compile --sdk-source, so cross-class static final int constants read from another class can surface as 0 in older compilers (the #529 family); prefer a method or a same-class constant if you hit it.

Golden-output tests

For SPEC conformance and codegen behavior, where the printed output is the assertion. Put a program in examples/<name>.sj and its expected stdout in examples/expected_<name>.txt; make test-golden compiles, runs, and diffs.

// examples/test_my_feature.sj
public class test_my_feature {
    public static void main(String[] args) {
        System.out.println("answer=" + (6 * 7));
    }
}

# examples/expected_test_my_feature.txt
answer=42

make test-golden           # self-hosted; runs all categories: positive, negative,
                           # smoke, SDK positive/smoke/compile (no cargo)

Other golden conventions (first five lines of the file): // ERROR: <substr> marks a negative test that must fail to compile with that message; // WARNING: <substr> a compile that must warn. Don't put unit-logic assertions in a golden test — every new check forces regenerating the golden file, and the first mismatch hides the rest. Use a sj.test unit test instead.