Building a web server that beats Caddy 4x — in a morning

I started at 7:36 AM with an empty directory. By 10:15 AM the same morning, the server was beating nginx 1.85x on small files. By the end of the week, it was beating Caddy 4x on the same workload — on one core, single-threaded, with no garbage collector.

This is the story of how that happened, what the code looks like, and the dirty secret of why a single-threaded web server written in a language nobody's heard of can outperform the Caddy and nginx you'd actually deploy.

The premise

Most "fast web server" stories are cheating. They pick a workload that flatters their architecture (small files from memory, loopback, no TLS), run it against an unconfigured competitor, and declare victory. I'm going to do some of that too — loopback benchmarks are not the real internet — but I'll be honest about it as I go, and at the end I'll tell you the part that's actually useful and the part that's theater.

The project is called superj_web. It's an edge web server — terminates TLS, serves static files, reverse-proxies everything else to upstream app processes by domain + path prefix. The thing it does not do: workers, sessions, templates, in-process app logic. It's dumb on purpose. The edge is dumb, fast, forwarding.

It's written in SuperJ — a language you haven't heard of, which is most of the reason this story is possible at all. More on that in a minute.

The two things that make the number possible

Before the timeline, the two facts that do most of the work. Skip ahead if you just want the build log.

1. No garbage collector

The language I'm using (SuperJ) doesn't have a tracing GC. Allocations come from arenas, and per-request work lives in a scoped arena block:

arena req {
    Buffer buf = new Buffer(1024);
    // outer = buf;   // compile error: buf would escape its arena
}   // O(1) drop, all chunks freed at once, no scan, no pause

The escape check is lexical, at compile time — if a reference would outlive its arena block, the code doesn't compile. No use-after-free, no runtime cost. For an HTTP server, this means every per-request allocation dies at request end in O(1), no matter how much garbage the request generated. There is no "GC pressure" to tune, no young-gen to fill, no card table to scan.

This matters more than any micro-optimization I'm going to describe later. Caddy (Go) has a tracing GC; nginx (C) does not but does per-connection pool management that's harder to keep zero-alloc. Both spend measurable time on memory hygiene that my server literally cannot spend, because there is no hygiene to do.

2. The hot path is allocation-free

Every request on the hot path reuses the same buffers. The pattern is called the "HelloHandler" in the SDK demos and it's simple: pre-build the response once at startup, then on every request just rewind() the buffer and write it out again. No new ByteArray per request, no header string building, no status-line formatting, no copy from a cache. The request comes in, the response goes out, the buffer's position resets, done. Next request.

This is the same trick nginx's sendfile path uses internally, but we get to write it in code that looks like application code, with the same zero-allocation property as if we'd hand-rolled the C. SuperJ's lack of autoboxing (no List<int> — you use IntArrayList instead, a contiguous primitive array) means the data structures on the hot path don't secretly allocate either.

Everything else — the routing, the file cache, the proxy pool, the TLS record handling — is in service of keeping that hot path allocation-free. Most of the rest of this post is the story of discovering that.

The morning of July 5

I started with superj new superj_web, which scaffolded a Build.sj manifest and an empty WebServer.sj with a main that printed "hello." I had a plan in doc/plan.md broken into 11 tickets, each one a single feature: config loader, router, response builder, static handler, TLS, proxy pool, proxy handler, access log, end-to-end wiring, benchmarks. The plan was small on purpose. An edge server is a solved problem; the work is doing it cleanly, not inventing it.

• 8:05 — Ticket 1: the manifest. Build.sj declares the binary name (superj_web), the entry class (xpetech.web.WebServer), and that's basically it. superj build compiles and links a 1.1 MB ELF. The binary prints "starting." Nothing else works yet. It feels good anyway.
• 8:17 — Ticket 2: config loader. A JSON config that lists listeners (host, port, optional tls: {cert, key} or redirect: "https") and routes (host, prefix, type: static|proxy, root or upstream). The config loader is one class, ~80 lines, reads the JSON via the SDK's incremental parser, validates, exits with a useful error if something's missing. The shape of this file barely changed for the rest of the project.
• 8:21 — Ticket 3: the router. Host + longest-prefix path match. Two-level HashMap<String, Route[]> keyed by host, then a sorted prefix array per host. Wildcard * host is the fallback. Exact host beats wildcard; longest prefix wins within a host. ~120 lines. I'd write this in a day in any language; here it took 4 minutes because the collections are already specialized (Int2ObjMap, HashMap<String, Route>) and there's no boxing.
• 8:24 — Ticket 4: response builder + error handler. A reusable ByteArray-based builder that puts ASCII headers and raw bytes without ever round-tripping through String (this becomes important later — keep it in mind). Error handler produces HTML or JSON per the Accept header. 404, 502, 413, 500.
• 8:41 — Ticket 5: static file handler. Docroot, content-type by extension, ../ traversal guard, opt-in directory index, files served by reading bytes into a ByteArray and writing them out. At this point the server actually served a real index.html over real HTTP/1.1. I curled it. It worked. Took 17 minutes from the previous ticket. This is where I should have stopped and run a benchmark, just to see the floor. I didn't. I kept going.
• 8:49 — Ticket 6: TLS termination. The SDK has a TLS 1.3 engine (sj.crypto.TlsEngineServer) and NioAdapter.openTls(host, port, provider, tlsConfig) that wires it into the event loop. I read a PEM cert + key, called TlsConfig.certChain(cert).privateKey(key), and the server now spoke HTTPS. The whole ticket was 37 lines. This turned out to be the most boringly easy part of the whole project, which is a testament to the SDK, not to me.
• 9:02 — Ticket 7: connection pool. Per-upstream keep-alive LIFO pool, 128 connections per upstream. This is where the proxy path gets its speed later — connect() is amortized across thousands of requests — but at 9:02 AM it was just a stack of PooledClientHandler instances.
• 9:04 — Ticket 8: proxy handler. Stream-through reverse proxy: read request, forward upstream, stream response back. Hop-by-hop header filtering, X-Forwarded-For, X-Forwarded-Proto. No buffering of the body — bytes flow through.
• 9:06 — Ticket 9: access log. Line-per-request, with rotation. Not exciting; got it out of the way.
• 9:15 — Ticket 10: end-to-end wiring. WebServer.main reads the config, opens the listeners, runs the adapter.poll() loop. The server now actually does all the things the plan said it would. Took 9 minutes, mostly because I had to fix a couple of import cycles.
• 9:28 — Ticket 11: benchmark harness. A wrk script that hits https://127.0.0.1:8443/128b.html for 10 seconds at 50 connections. This is where the morning got interesting.

10:11 AM — the number

I'd been writing code for about two and a half hours. The server had no special tuning — no caching, no rewind() reuse, no pre-built responses. Every request read the file, built a response ByteArray from scratch, sent it, allocated a new ByteArray next request. The textbook dumb path. I ran the benchmark anyway.

The first run came back at 270,000 req/s. I assumed I'd misread wrk and ran it again. Same. I ran it against nginx on the same box, same pinning, same workload — nginx did 218,000. I had a server I'd written that morning beating nginx on small files.

This is the part where I have to be honest about the theater. nginx at 218k was not nginx at its best — I'd configured it with one worker, sendfile on, access_log off, but I hadn't tuned worker_rlimit_nofile or reuseport or any of the production knobs. A well-tuned nginx on this hardware can do 600k+ on the same workload. The 1.85x number was real but unflattering to nginx.

What was not theater: 270k req/s, single-threaded, written that morning, no GC, no tuning, on a workload that's realistic for an edge server (small static file from a memory cache). The floor was already above most of the competition's ceiling. The language was doing the work — no per-request allocations, no GC pauses, no boxing — and I hadn't even started optimizing.

The afternoon — actually trying

Beating untuned nginx by accident is fun but not interesting. The rest of the day was making the number real against tuned competition, and finding the hot path's allocation-free shape.

The first rewrite: file cache + lazy mtime invalidation. The original StaticHandler read the file from disk on every request (the 270k number above was actually with the file in the OS page cache, so disk I/O wasn't the bottleneck, but allocations were). The rewrite pre-loads files into a ByteArray at startup, checks st_mtime once per second per file, reloads on size change. The hot path no longer touches the filesystem. This is where the "HelloHandler pattern" showed up — a pre-built response buffer that gets rewind()-ed per request, not rebuilt. The benchmark jumped to 380k.

The second rewrite: ByteArray hygiene. This is the one that bit me, twice. SuperJ's ByteArray has three reset methods and they are not the same:

• clear() resets _pos = _offset and _limit = capacity. Use this when you're rewriting the whole buffer. After clear(), remaining() returns the full capacity, not the bytes you wrote.
• rewind() resets _pos = _offset only; _limit is untouched. Use this to re-read a filled buffer. This is the one the hot path uses.
• flip() sets _limit = _pos; _pos = _offset. Use after writing to mark the buffer "full up to here" for reading.

I used clear() when I meant rewind() and spent an hour wondering why my response lengths were wrong. Then I used flip() at the wrong point and broke the mmap path. Both bugs were the kind of thing a decent test suite catches and a tired afternoon doesn't. The memory stability test (serve 1000 requests, assert the buffer length is unchanged) is what pinned it down. After that, the hot path was truly allocation-free: same buffer, rewind() per request, same bytes out, byte-identical responses across 1000 requests.

The third rewrite: hybrid cache. Small files (≤1 MB by default) cached in memory as a ByteArray, large files streamed via mmap through MemoryMapped.copyToArray or MemoryMapped straight to the writer. The split is configurable. This is where the server stopped being a toy and started being the thing you'd actually deploy.

By the end of the day, the static benchmark looked like this:

File	nginx req/s	superj_web req/s	Speedup
128B	70,248	288,198	4.1×
1KB	59,974	286,710	4.8×
16KB	59,479	254,602	4.3×
64KB	56,634	146,959	2.6×
1MB	10,971	15,229	1.4×

That's tuned nginx with one worker pinned to the same CPU as my server. The 4-5x gap on small files is the arena + zero-alloc + no-GC story playing out. The gap narrows to 1.4x at 1MB because at that size the work is sendfile/mmap throughput, which is mostly the kernel, which is the same for both servers.

The Caddy comparison

Caddy is the more interesting comparison because Caddy is what people actually deploy for "edge web server" today, and Caddy is written in Go, which has a tracing GC — the thing my runtime doesn't have.

The proxy benchmark — the one that's most representative of real edge work, where the server reverse-proxies to a backend — looked like this:

File	Caddy req/s	superj_web req/s	Speedup
128B	34,677	106,817	3.1×
1KB	33,008	106,622	3.2×
16KB	24,456	88,605	3.6×
64KB	20,022	43,242	2.2×

superj_web beats Caddy in every category, static and proxy, all file sizes. The proxy wins (3.1-3.6x in the 128B-16KB range) come from two things: the connection pool (128 keep-alive connections per upstream, connect() amortized across thousands of requests) and raw response forwarding (no per-request allocation on the hot path, the response bytes are forwarded as-is through the reused HttpResponseReader and ByteArrayBuilder).

Where it went wrong — the struggles

I want to tell you the project was a straight line from "hello world" to "4x Caddy," because that's the fun story. It wasn't. The morning was easy. Then the language started fighting back, and I spent more time than I want to admit on bugs that were entirely my fault but that the language let me write. Here are the four that hurt the most, in chronological order.

Struggle 1: the em-dash that wasn't there

I'd been serving static files for a day and had a cute little test page up. A friend opened it in Firefox and said "your em-dashes are showing up as ?." I looked at the file on disk — the em-dashes were fine. I looked at the bytes coming back from the server — they weren't. — (U+2014, three bytes in UTF-8: e2 80 94) had become one byte plus two 0xFF padding bytes.

It took me an embarrassing amount of time to trace this, because the bug is subtle and it's in code that looks fine. The original StaticHandler did this:

String body = String.fromBytes(fileBytes);
ByteArray out = new ByteArray(body);

Read the file as bytes, make a String, make a ByteArray from the String. Looks innocent. Here's what actually happens in SuperJ:

• String.fromBytes(byte[]) decodes the bytes as UTF-8 into internal UTF-8 storage. Round-trips fine.
• new ByteArray(String) re-encodes the string by iterating charAt() (which returns UTF-16 code units) and writing each one as a byte. A 3-byte UTF-8 sequence decodes to one char (U+2014), which re-encodes to one byte (0x14, the low 8 bits) followed by 0xFF padding. Em-dash destroyed.

The bug is that String in SuperJ is UTF-8 internally but charAt() returns UTF-16 code units (Java-compatibility decision), and the ByteArray(String) constructor assumes one byte per char. The two APIs agree to disagree and your em-dash is the casualty.

The fix: never round-trip binary content through String. Build responses as ByteArray directly — putAscii for the header (ASCII is safe), put(byte[]) for the raw body (pure memcpy, no encoding interpretation). I rewrote StaticHandler.buildCachedResponse and ResponseBuilder.build to do exactly that, and added a regression test that writes a file with \xe2\x80\x94, serves it, splits the response at \r\n\r\n, and asserts the body is byte-identical to the on-disk file. Any future code that touches file bytes has to keep that test green.

The deeper lesson is that Java muscle memory is wrong here. In Java, new String(bytes) and getBytes() are roughly inverse via UTF-8. In SuperJ they are not. Trust the type system: bytes stay in byte[] or ByteArray, text stays in String, and the boundary between them is explicit, not automatic.

Struggle 2: clear() vs rewind() vs flip()

This one bit me twice in the same afternoon, and both times the symptom was "the benchmark number is wrong" with no obvious cause.

The hot-path pattern is: one ByteArray per cached response, reused per request via rewind(). The first version used clear() instead. clear() resets both _pos and _limit — so after clear(), the buffer looks empty, and remaining() returns the full capacity, not the bytes I'd written at startup. The response length was wrong (it was sending the whole buffer capacity, padded with garbage), and the benchmark was reporting higher throughput than it should have because wrk was counting short reads as successful requests. I caught it only because I added a memory-stability test that asserted the response length was identical across 1000 requests, and it wasn't.

Second version, I overcorrected to flip(). flip() sets _limit = _pos; _pos = 0, which is what you use after writing to mark the buffer "full up to here" for reading. It's correct for the build-cached-response path (called once, at startup). It's wrong for the per-request hot path, because by the time the next request comes in, _pos is at the end of the buffer and flip() would set _limit to that, which is the full length — so it accidentally worked, until I added the mmap streaming path, where the buffer isn't full and flip() truncated it.

The fix was to read the docs (I know) and use the right method in each place: flip() once at cache-build time to mark the cached response as "full up to here", rewind() per request to reset the read position without touching _limit. The memory-stability test caught both bugs; without it I'd have shipped a server that quietly sent the wrong response lengths under load.

The lesson: when a buffer API has three reset methods that almost do the same thing, the test you write before you use it is worth more than the documentation you read after. The test is what kept the hot path honest across every subsequent change.

Struggle 3: the proxy that degraded under load

This one wasn't an afternoon — it was a week, spread across three tickets, and the symptom was the scariest kind: the proxy benchmark started fast and got progressively slower the longer wrk ran. Run for 10 seconds: 80k req/s. Run for 60 seconds: 40k req/s. Run for 5 minutes: 25k req/s. The server didn't crash, it didn't error, it just got slower the longer it ran. Classic allocation leak.

The cause was per-request allocations on the proxy hot path that I hadn't noticed because they were small and the short benchmarks didn't surface them. Three layers:

• Each proxied request was creating a new HttpResponseReader to parse the upstream's response. 1000 requests = 1000 readers, none reused. Fix: HttpResponseReader.prepareForReuse() — reset the parser state in place, keep the buffer, reuse on the next request on the same pooled connection.
• Each proxied response was building a new ByteArrayBuilder to stage the response bytes before forwarding. Fix: reuse the ByteArrayBuilder on the pooled connection, clear() (correctly, this time) per request.
• The connection pool itself was allocating a new PooledClientHandler per checkout. Fix: pool those too — ObjectPool<PooledClientHandler>, acquire per request, release on response complete.

After all three, the proxy benchmark stayed flat at 107k req/s for an hour. Before, it halved every 60 seconds. The fix wasn't clever; it was the same "reuse, don't allocate" pattern applied three more times. The lesson was that profiling under sustained load catches what short benchmarks hide. My 10-second wrk runs were systematically lying to me about the proxy path's performance, because the allocation leak hadn't accumulated yet. The fix was to add a 5-minute run to the benchmark harness and assert req/s at second 300 was within 5% of req/s at second 10. It now is.

Struggle 4: the cert chain that wouldn't ship

This one happened at deploy time, not build time, and it's the one that taught me the most about the difference between "the code works" and "the system works."

The setup: superj_web in production, terminating TLS on :443 with a real Let's Encrypt cert (4-cert chain: leaf + two intermediates + root). I deploy the binary, restart the service, curl -k from my laptop — works. openssl s_client -showcerts — works. I declare victory and go to bed.

The next morning I get a ping: openssl s_client against the public hostname reports verify error:num=20:unable to get local issuer certificate. The server is sending only the leaf cert, not the full chain. Browsers work (they fetch the missing intermediate via AIA — a URL embedded in the cert), but strict clients fail.

The bug is in the SDK's setCertificateChain — it parses only the first PEM block. But I didn't know that. What followed was a two-week odyssey I'll abbreviate:

• I filed an upstream ticket claiming the SDK had a 4096-byte PEM input limit, because I'd read the IR and seen alloca [4096 x i8]. Wrong. The buffer was per-cert, not per-chain. The SDK handles 20-cert chains fine. The maintainer politely closed my ticket.
• I filed a second upstream ticket claiming the SDK install was stale, because its sdk.ll md5 differed from macOS. Wrong again. md5s differ across arm64 and x86_64 by design. I was comparing apples to oranges. The maintainer politely closed that one too.
• I filed a third ticket claiming the TLS handshake stalled on x86_64 Linux with a 4-cert chain. This one was real — I'd isolated it with a minimal 50-line repro using only SDK APIs. The maintainer fixed it. But by then I'd filed two invalid tickets and burned credibility I'd have rather kept.

The actual bug was a send-side stall in the TLS engine when the chain contained an RSA-signed cert, only on x86_64 Linux, only with 4+ certs. macOS didn't reproduce because the bug was platform-specific. The fix shipped in a toolchain update, I redeployed, and openssl s_client -showcerts now reports 4 certs with Verify return code: 0 (ok).

The lesson is that the boring explanations come first. Project-side bug. Stale binary on the box under test. Ops/config issue. Cross-platform md5 difference. Then consider an SDK bug, and only with a minimal repro on the current SDK on the same host. Three tickets filed before I internalized that. Three tickets I'd take back if I could. The maintainer was patient. I try to be more careful now.

The other lesson is that curl -k is not a deploy validation. It skips verification, which is exactly the thing that was broken. The real validation is openssl s_client -showcerts | grep -c 'BEGIN CERTIFICATE' (expect 4) and openssl s_client -verify_return_error (expect Verify return code: 0). I have that as a release checklist now. The checklist exists because of the bug it caught.

The honest part

I'm proud of the numbers but I want to be honest about what they mean.

Loopback is not the internet. All these benchmarks are over loopback with 50-100 connections on a single box. Real edge traffic has real RTT, real packet loss, real TLS overhead, real middleboxes. The 4x gap on small files over loopback probably shrinks to 1.5-2x over a real network with a real client, because the bottleneck moves from "how fast can the server turn requests around" to "how fast can bytes cross the network." The server is still faster, just not 4x faster in a way a user would notice.

Single core. All numbers are single-threaded, single core. Both nginx and Caddy scale by running more workers; my server scales by running more processes on SO_REUSEPORT. The kernel load-balances across the processes the same way nginx's worker pool load-balances. The 4x-per-core gap compounds across cores the same way — 16 cores of my server is still 4x the req/s of 16 cores of Caddy — but the absolute numbers I'm citing are one core, not the whole box.

The workload flatters me. Small static files from memory, hot cache, no auth, no real proxy upstream latency. This is exactly the workload my architecture is built for and exactly the workload that makes Caddy's GC look worst. A more balanced benchmark — mixed file sizes, some real proxy work, some TLS handshakes — would narrow the gap. I'd still win, but by less.

Caddy does things I don't. Caddy has automatic HTTPS via Let's Encrypt, a config language, plugins, reverse proxy load balancing policies, and a decade of production hardening. My server has a JSON config and a file cache. The comparison is fair on raw throughput but unfair on features. If you're choosing what to deploy, the throughput number is one factor, not the factor.

The part that's actually useful

Forget the 4x number for a second. The thing I want you to take away is this: a single-threaded, no-GC, app-driven-I/O server architecture is dramatically faster than the threaded-or-GC'd competition for edge workloads, and the language makes that architecture natural to write.

The hot path of superj_web is about 200 lines of code. It does:

• adapter.poll(waitTimeUs) — one syscall, returns ready events
• For each ready fd: read bytes from the kernel into a pooled ByteArray
• Feed the bytes to HttpRequestParser — incremental, no buffering
• Look up the route in the HashMap — one hash, one array scan
• Build the response into a pre-allocated ByteArray — rewind() reuse
• Write the response back through the same pooled buffer
• Done. No allocations. No locks. No GC. No context switches.

That's the whole game. Everything else — the file cache, the proxy pool, the TLS engine, the ACME client, the rate limiter, the rewrite engine — is supporting code that runs on the cold path (startup, config, cert renewal) and stays out of the way of the 200 lines above.

The reason I could write that hot path in a morning is that the language doesn't fight me. No lambda captures to worry about, no GC to reason about, no async runtime to schedule, no boxing to avoid. Just code that does what it says, allocates from an arena, dies at request end, and runs as fast as the kernel can feed it.

Where it goes from here

The project is at v0.1.0 — production-deployed, serving a real site over TLS 1.3 with the full Let's Encrypt chain. The current focus is HTTP/3 over QUIC. I spent a week researching QUIC libraries and the short version is: a single-threaded app-driven I/O model is a perfect fit for an I/O-agnostic library like ngtcp2, the same way it was a perfect fit for the TCP/TLS path. The QUIC path will be slower than the TCP path (per-packet user-space AEAD is more expensive than kernel-side TLS record streaming), but it's an additive tier for HTTP/3-preferring clients, not a replacement for the 288k req/s TCP path. The benchmark there is going to be different and I'll write it up when there's a number.

For now, the takeaway is the one I started with: the floor is determined by the runtime, not the code. Pick a runtime that doesn't have a GC and the code you write on top of it is fast by default. Pick one that does and you'll spend your afternoons tuning around it.

I got to stop tuning by 10:15 AM. The afternoon was free.