Per-line SHA-256 (FFI overhead)
Compute a 32-byte content digest of Frankenstein (Project Gutenberg #84, ~7,700 lines) by hashing each line with the runner's bundled SHA-256 primitive and XOR-folding into a running accumulator. Measures interpreter throughput + per-call native FFI overhead in equal measure.
Compile time · median (cold)
Full statistics
| Runner | N | Compile | Runtime | P95 | Stddev | RSS | vs pipit | Status |
|---|---|---|---|---|---|---|---|---|
| Native Gocompiled | 7 | 120 ms | 2.01 ms | 2.01 ms | 1.51 µs | 1.62 GiB | 110× | OK |
| Pipitbytecode VM | 7 | 1.09 ms | 31.7 ms | 49.6 ms | 7.09 ms | 592 MiB | 1.00× | OK |
| CPython 3.14bytecode VM | 7 | 310 µs | 47.9 ms | 48.3 ms | 489 µs | n/a | 0.28× | OK |
| PyPy 7.3tracing JIT | 7 | 247 µs | 22.4 ms | 23.8 ms | 680 µs | n/a | 0.23× | OK |
| tengobytecode VM | 7 | 189 µs | 79.9 ms | 94.8 ms | 6.32 ms | 1.69 GiB | 0.17× | OK |
| scriggobytecode VM | 7 | 421 µs | 81.6 ms | 100 ms | 11.4 ms | 1.69 GiB | 0.39× | OK |
| mvmbytecode VM | 7 | 315 µs | 83.8 ms | 84.6 ms | 497 µs | 1.69 GiB | 0.29× | OK |
| yaegiAST walker | 7 | 262 µs | 55.7 ms | 71.4 ms | 7.08 ms | 1.69 GiB | 0.24× | OK |
Workload & symmetry rules
Workload
Walk Frankenstein's text (~441 KiB, ~7,700 lines) byte-by-byte to find newline boundaries. For each line, call the runner's bundled SHA-256 primitive once. XOR-fold every 32-byte digest into a running accumulator. Emit the accumulator as 64 lowercase hex characters.
Symmetry rules
- One SHA-256 call per line (~7,700 native FFI dispatches per inner iteration). Whole-buffer hashing (
sha256.Sum256(corpus)/hashlib.sha256(corpus).hexdigest()) is banned because it would do the work in a single native call, hiding the per-call signal. - Line scan and XOR fold run in interpreted code on every runner. No
strings.Split, nostr.splitlines. - Hand-rolled hex emission (no
encoding/hex, nobinascii.hexlify) so the trailing format step also runs in the interpreter. - Tengo's stdlib does not bundle SHA-256, so the harness registers a
sha256_sumbuiltin that wraps Go'scrypto/sha256.Sum256. Same shape of "register a host function as a native callout" that every other runner uses, just via Tengo'sUserFunctionsurface.
Why this benchmark exists
This is the only benchmark that exercises the script-orchestrates-compiled-primitive pattern, which is how most production embedded interpreters are actually used (NumPy from Python, engine bindings from Lua, host functions from Tengo). Per-call FFI dispatch overhead is what discriminates runners: with ~7,700 calls per inner iteration, a 1µs-per-call difference adds up to ~8 ms of wall time, visible against a ~100 ms baseline. Every runner uses the same SHA-256 algorithm routed through the same compression primitive (OpenSSL on Python, Go's crypto/sha256 with ASM where available on Go family), so the per-block hash cost is comparable across the board.
Source code
pipit / Go
pipit_source.gonative Go
native_main.goCPython / PyPy
cpython.pytengo
script.tengo