Benchmarks

What ran, and how fast

One row is one run: a machine, a model, a quant, a context length, and the software that served it. Generation tokens per second and prompt tokens per second are both recorded, because a long prompt and a short answer are different jobs.

Machine
The lab name, plus enough hardware to tell two boxes apart.
Model
The model that ran, not a size class.
Quant
The quantization of that file.
Size
Parameter size or on-disk size. Say which in the notes if it is ambiguous.
Context
The context length of the run.
Gen tok/s
Output tokens per second.
Prompt tok/s
Prompt-processing tokens per second.
RAM
What the run used, or installed RAM. Say which in the notes.
Date
The day of the run.
Software
Runtime and version. The notes sit under it.

Lab runs. Generation tok/s is the median of three runs. Prompt tok/s is the first uncached prefill. n/a means that pass was already cached, so it is not listed.
MachineModelQuantSizeContextGen tok/sPrompt tok/sRAMDateSoftware
Frankenstein qwen3:4b-instruct-2507-q4_K_MQ4_K_M4.0B dense, 2.5 GB204812.36n/a3.1 GiBNov 18, 2026Ollama 0.35.0
CPU only, num_gpu=0, Ollama default threads. Short prompt (~110 tokens). Clean RSS. Uncached prefill was not captured.
Frankenstein qwen3:4b-instruct-2507-q4_K_MQ4_K_M4.0B dense, 2.5 GB81926.6252.64.9 GiBNov 18, 2026Ollama 0.35.0
CPU only, num_gpu=0. Long prompt. Prefill is the first calibration pass (~3552 tokens), approximate. Clean RSS.
Frankenstein qwen3:8bQ4_K_M8.2B dense, 5.2 GB20486.91n/a10.4 GiB observedNov 18, 2026Ollama 0.35.0
CPU only, num_gpu=0, default threads. Short prompt. RSS includes a leftover 4B. Uncached prefill was not captured.
Frankenstein qwen3:8bQ4_K_M8.2B dense, 5.2 GB81924.6631.312.3 GiB observedNov 18, 2026Ollama 0.35.0
CPU only, num_gpu=0. Long prompt (~4012 tokens). Prefill is the first uncached run. RSS includes a leftover 4B.
Frankenstein qwen3:14bQ4_K_M14.8B dense, 9.3 GB20484.41n/a9.4 GiB cleanNov 18, 2026Ollama 0.35.0
CPU only, num_gpu=0, Ollama default threads. Short prompt. This is the default-thread cell. During the run RSS was 21.7 GiB with other models still loaded. 9.4 GiB is the later peak with 14B alone. Uncached prefill was not captured.
Frankenstein qwen3:14bQ4_K_M14.8B dense, 9.3 GB81923.0217.123.8 GiB observedNov 18, 2026Ollama 0.35.0
CPU only, num_gpu=0. Long prompt. Prefill is the first uncached run. RSS on run 1 includes a leftover 4B.
Frankenstein qwen3:14bQ4_K_M14.8B dense, 9.3 GB20484.24n/a9.4 GiB cleanNov 18, 2026Ollama 0.35.0
CPU only, num_thread=24 (physical cores), short prompt. Median of three runs: 4.12, 4.24, 4.43. Uncached prefill was not captured.
Frankenstein qwen3:14bQ4_K_M14.8B dense, 9.3 GB20481.33n/a9.4 GiB cleanNov 18, 2026Ollama 0.35.0
CPU only, num_thread=48 (all SMT threads), short prompt. Median of 1.08, 1.33, 1.35. About 3.2× slower than 24 threads. Uncached prefill was not captured.
Frankenstein frank-qwen3-30b-64kQ4_K_M30.5B MoE (3B active), 18 GB20489.58n/a27.2 GiB observedNov 18, 2026Ollama 0.35.0
CPU only, num_gpu=0, default threads. Qwen3-30B-A3B weights. The tag only sets num_ctx 65536; this run used 2048. Short prompt. RSS includes a leftover 14B. Uncached prefill was not captured.
Frankenstein frank-qwen3-30b-64kQ4_K_M30.5B MoE (3B active), 18 GB81928.7353.228.5 GiB observedNov 18, 2026Ollama 0.35.0
CPU only, num_gpu=0. Long prompt (~4012 tokens). Prefill is the first uncached run. Context 65536 was not loaded. RSS includes a leftover 14B.
Frankenstein qwen3:4b-instruct-2507-q4_K_MQ4_K_M4.0B dense, 2.5 GB204869.57n/a3.1 GiB VRAMNov 18, 2026Ollama 0.35.0
Tesla P40, num_gpu=99. Short prompt. 5.6× the CPU generation rate. Clean VRAM. Uncached prefill was not captured.
Frankenstein qwen3:4b-instruct-2507-q4_K_MQ4_K_M4.0B dense, 2.5 GB819260.031,1004.9 GiB VRAMNov 18, 2026Ollama 0.35.0
Tesla P40, num_gpu=99. Long prompt. 9.1× the CPU generation rate. Prefill ~1100 is the first calibration pass; load and prefill may overlap. Clean VRAM.
Frankenstein qwen3:8bQ4_K_M8.2B dense, 5.2 GB204844.1839110.3 GiB VRAM observedNov 18, 2026Ollama 0.35.0
Tesla P40, num_gpu=99. Short prompt. 6.4× the CPU generation rate. Prefill ~391 is run 1 and approximate. VRAM likely includes a leftover 4B.
Frankenstein qwen3:8bQ4_K_M8.2B dense, 5.2 GB819240.2274812.1 GiB VRAM observedNov 18, 2026Ollama 0.35.0
Tesla P40, num_gpu=99. Long prompt. 8.6× the CPU generation rate. Prefill 748 is the first uncached run. VRAM may include a leftover 4B.
Frankenstein qwen3:14bQ4_K_M14.8B dense, 9.3 GB204825.4236421.3 GiB VRAM observedNov 18, 2026Ollama 0.35.0
Tesla P40, num_gpu=99. Short prompt. 5.8× the CPU generation rate. Prefill ~364 is run 1 and approximate. VRAM is 14B plus a leftover 8B, nearly the full 24 GB.
Frankenstein qwen3:14bQ4_K_M14.8B dense, 9.3 GB819221.8543411.1 GiB VRAMNov 18, 2026Ollama 0.35.0
Tesla P40, num_gpu=99. Long prompt. 7.2× the CPU generation rate. Prefill 434 is the first uncached run. 11.1 GiB is the settled VRAM for 14B alone.
Frankenstein frank-qwen3-30b-64kQ4_K_M30.5B MoE (3B active), 18 GB204865.2925517.8 GiB VRAMNov 18, 2026Ollama 0.35.0
Tesla P40, num_gpu=99. Short prompt. 6.8× the CPU generation rate. Prefill ~255 is run 1 and approximate. Clean VRAM. Context 65536 was not loaded.
Frankenstein frank-qwen3-30b-64kQ4_K_M30.5B MoE (3B active), 18 GB819257.6487118.9 GiB VRAMNov 18, 2026Ollama 0.35.0
Tesla P40, num_gpu=99. Long prompt. 6.6× the CPU generation rate. Prefill 871 is the first uncached run. Clean VRAM, about 5 GiB free on the 24 GB card. Context 65536 was not loaded.

On a narrow screen, scroll the table sideways. The machine name stays put.

Frankenstein, 18 Nov 2026

These rows are from Frankenstein: two Xeon E5-2697 v2 CPUs (24 cores, 48 threads), 288 GB of DDR3 RDIMM at 1067 MT/s, and a Tesla P40 24 GB, running Ollama 0.35.0. Generation speed is the median of three runs. Prompt speed is the first uncached prefill. Identical prompts after that hit Ollama’s prefix cache and are not in this table. The write-up is CPU-only on a used Xeon: what actually runs.

Add a row by editing src/data/benchmarks.json. The file is checked at build time. A measured table cannot keep a sample row, and a measured row needs a generation speed and a date. Leave prompt tok/s null when the uncached prefill was not captured. The methodology is the rule for what belongs in a cell.