Home · ← SuperJ Manual EN|中文

Memory Management Best Practices

SuperJ has no garbage collector. new allocates from an arena; the default arena is the global one, which lives until the process exits. That gives you allocation at bump-pointer speed and zero pause time — and one obligation in exchange: there is no such thing as a short-lived object unless you make it one. A new in a hot path is not "pressure the collector will absorb"; it is a leak with a rate.

This page is the practice: how to make memory usage flat — the ladder of allocation strategies, the arena and stack tools, the scratch-buffer cookbook, and the built-in log that proves your process is behaving.

Contents

  1. The key insight: single-threaded is a license
  2. The allocation ladder
  3. Leveraging arenas
  4. Scratch buffers: the reuse cookbook
  5. Stack allocation with local
  6. Watching it: the memory-event log
  7. The review checklist

1. The key insight: single-threaded is a license

SuperJ programs are strictly single-threaded (you scale with processes, not threads — see Concurrency). For memory this is not a limitation; it is a license that removes an entire category of cost that multi-threaded code pays:

Everything below is a technique for exploiting this license: the goal is a process whose memory footprint is flat no matter how many requests, queries, or iterations it serves.

2. The allocation ladder

When you are about to write new, try these in order:

  1. Don't allocate. Most allocations are avoidable outright: return a primitive instead of a wrapper; fill a caller-owned array instead of returning a fresh one; read two longs out of a mapped file instead of materializing an index object; precompute strings once instead of concatenating per call.
  2. local — stack-allocate a small, provably non-escaping temporary (§5). Freed on scope exit, costs nothing. Two sharp caveats below.
  3. Reuse a long-lived scratch buffer — for anything sized by the data, this is the answer (§4). Allocate once, reuse across every call; grow on demand, never shrink.
  4. An arena block — for a batch of scoped intermediates you want dropped together, in O(1), with escape checked at compile time (§3).
  5. The default heaps (what you get with no arena block) — only for things that genuinely live as long as the process: precomputed tables, the scratch buffers themselves, long-lived indexes and caches. "The global arena" is the usual shorthand, but the default is really three domains and only one of them is that arena — see §2.1 below.

Real numbers from a database project built on SuperJ, all of which were one innocuous-looking new in a loop: a materialized column per query — 22.8 GB peak RSS; a sorted table copy per call — 1.3 GB per call; a path string per column per partition — unbounded growth with query count. Every one was fixed by moving one rung up the ladder.

2.1 The default is three domains

Rung 5 is usually called "the global arena", and that is a useful simplification until you try to measure it. Outside any arena {} block, which allocator you get depends on what you allocate:

what you writeallocatorfreed?seen by System.memReservedBytes()
new Obj(...)callocneverno
new T[n]sj_galloc → the global arenaneveryes, but only when a new chunk is reserved
a String body ("a" + n, substring, …)mallocneverno

Measured from emitted IR in all three compile modes (standalone, prebuilt-archive, whole-program source) — the split is the same in each.

All three are equally permanent, so the ladder's advice does not change. What changes is what you can observe:

So a flat counter is not evidence that nothing is leaking. Build with --mem-track when you are chasing growth (§6) — that turns on the per-domain counters, including the malloc domain, which is compiled out otherwise.

Inside an arena {} block all three converge on that block's arena — objects, arrays and strings alike — and all of it is reclaimed on drop. That is why block allocations move the counter and default ones do not.

3. Leveraging arenas

An arena block gives a batch of allocations a shared, scoped lifetime:

int result;
arena temp {
    Buffer buf = new Buffer(1 << 20);   // allocates from temp
    Index idx  = new Index(buf);        // so does this
    result = idx.lookup(key);           // primitives escape freely
}   // temp dropped here — O(1), all chunks freed at once, no scan, no pause

What makes it the checked tool:

Use an arena block when the intermediates are genuinely a batch with one lifetime — a parse, a request, a query plan. For a buffer that every call needs, don't re-create it per block; move it up to a scratch (§4).

4. Scratch buffers: the reuse cookbook

The workhorse of flat memory. Allocate once (global arena — rung 5 paying for rung 3), grow on demand, never shrink, reuse forever:

static double[] scratch = null;

static double[] ensureScratch(int n) {
    if (scratch == null || scratch.length < n) scratch = new double[n];
    return scratch;
}

The patterns that matter in practice:

File reads into a caller-owned buffer. Reading a whole file into a fresh byte[] per call leaks the file size per call. Size one scratch to the largest file in the batch, then reuse it for every read:

long maxLen = 0;
for (int i = 0; i < paths.length; i++) {
    long sz = new File(paths[i]).length();
    if (sz > maxLen) maxLen = sz;
}
byte[] scratch = new byte[(int) maxLen];
for (int i = 0; i < paths.length; i++) {
    columns[i] = openColumn(paths[i], scratch);   // fills, never allocates
}

The explicit-count rule. A scratch buffer is only guaranteed to be >= n — it usually carries a previous call's data past the current length. Any method handed a possibly-oversized buffer must also take an explicit count, and use it, never buf.length. Using buf.length where you mean "row count" silently reads (or writes!) the previous caller's data — this class of bug has shipped a corrupt database.

Reused objects get reset(), not new. A builder or parser that runs per request keeps its internal buffers and exposes reset(); constructing a fresh one per request rebuilds capacity from zero and leaks the old one.

One scratch per purpose, sized by the maximum. Don't pool many small buffers; keep one that has seen the largest input. Memory stays bounded by the biggest item ever processed, not by the number processed — which is exactly the flatness you want.

Size working sets to the cache, not to the data. When streaming, a small reused window (tens–hundreds of KB) is both the constant-memory answer and the fast one — the produce and consume passes hit the same cache-resident bytes. Measure the window size; don't guess it.

Don't write a static scratch inside a hot loop on x86. A static scratch that is read per iteration (a lookup table, precomputed coefficients) is exactly right. A static scratch that is written per iteration and then fed to a SIMD intrinsic is an aliasing barrier: the store to a fixed global address can't be proven not to alias the arrays you read, so the optimizer must genuinely store every lane to memory and reload it — a round trip scalar-replacement cannot eliminate. Measured on a 20M-row kernel: static final double[4] scratch → Simd.dotDouble4 cost 4.5× the identical loop with a method-local double[4], which the optimizer SROA-promotes to registers. The static pattern is the one this page recommends for avoiding per-call allocation — and for scratch that is filled once and reused, it is correct. It only bites when the scratch is filled and consumed every iteration as a staging buffer for a SIMD call. For that shape, prefer scalar accumulators (as fast as the local array, allocate nothing) or declare the staging array inside the method. This is x86-only: on ARM/NEON the static version is the fastest variant, so developing on Apple Silicon gives no signal — measure on the target architecture.

5. Stack allocation with local

local Point p = new Point(1, 2);     // alloca — freed on scope exit
local int[] buf = new int[4096];     // stack — size is a literal

local puts a small temporary on the stack: zero arena traffic, freed on scope exit. It needs an initializer and is not allowed on primitives (a plain int is already a register). Two caveats, both of which can silently cost you the entire benefit:

Keep them small — a local int[1 << 20] is 4 MB of stack frame. And know that at release optimization levels the compiler already scalar-replaces simple non-escaping objects: new Point(a, b) in a loop is often free whether or not you write local. It earns its keep on shapes the optimizer can't fold — measure before assuming it did anything.

6. Watching it: the memory-event log

The proof of good memory management is a flat peak RSS across iteration counts. SuperJ ships the instrument (full guide: Memory-Event Log):

# 1. Compile with the arena instrumentation armed
superj compile myapp.sj --sdk-path "$SJ_HOME/sdk" --link --mem-track --output myapp

# 2. Run with the log enabled (binary, mmap-backed, survives SIGKILL)
SJ_MEM_LOG=myapp.mlog ./myapp

# 3. Read it
superj memlog myapp.mlog

The log records arena create/expand/drop events with source locations and timestamps — so growth is attributable: you see which allocation site's arena keeps expanding, even in an OOM post-mortem. The default build pays zero cost; --mem-track without SJ_MEM_LOG is armed-off and near-free, so you can leave it in deploy builds and enable logging per run.

The acceptance test to adopt: run your workload at a low and a high iteration count and compare peak memory. Flat is the pass condition, not "small" — if peak grows with iterations, something in the loop still allocates, and the log will name it.

7. The review checklist

Reading a diff (yours or anyone's), these questions catch nearly everything:

Then measure rather than assume: compare peak RSS at two iteration counts, and read the memlog when it isn't flat.

SuperJ — manual · generated from memory-management.md at pack time EN|中文