Benchmark without fooling yourself

Current performance status

RegesARC 0.0.1 is a functional baseline using ordinary layer offload. It has no established speed advantage over LM Studio. MoE-specific placement experiments are under investigation and are not shipped as the native default. The documentation does not turn a single generation into a “faster” claim.

Exploratory placement observation

On 20 October 2026, a private Lemonade comparison kept the same RegesCore 35B MXFP4 file, 8,192 context, temperature 0, one slot and an eight-thread CPU configuration. Ordinary 12-layer offload was compared with all eligible GPU layers plus CPU-routed experts (--cpu-moe). All four requests hit a 64-token reasoning budget before a final answer; this is throughput evidence, not answer correctness.

Strategy	First-request decode	Repeated-prefix decode
Ordinary 12-layer offload	0.14 tokens/s	0.38 tokens/s
GPU layers + CPU-routed experts	2.25 tokens/s	3.48 tokens/s

The machine was an i9-9900K/roughly 80 GiB RAM/RTX 4070 SUPER 12 GiB system. Decode counted completion tokens after the first content event, including reasoning. There were only two sequential requests per strategy, A always preceded B, a 36-token prompt prefix was reused, and no explicit seed or randomized ABBA order was used. Load timings were not cold comparisons. There was no LM Studio, BF16 or quality-parity comparison.

The guarded CPU-expert option subsequently completed actual installed functional qualification, including a final answer, cancellation and recovery. That functional check does not expand this exploratory throughput protocol. It is CPU expert computation, not Strata or a predictive GPU cache. It does not establish every file’s compatibility or speed. The default remains the ordinary baseline until a separately qualified release says otherwise.

Keep the comparison controlled

Use the same immutable artifact, prompt bytes, chat template, tokenizer, context, sampling, output limit and workload. Record engine/build, driver, hardware, placement strategy and concurrent machine activity. If the artifact changes, label that as an artifact comparison rather than an engine improvement.

Test one owned inference process at a time. Close competing model workloads deliberately. A package compression job, browser WebGL animation or different power/thermal state can contaminate results. Repeat runs and disclose the protocol instead of selecting the fastest anecdote.

Measure distinct phases

Metric	What to record
Cold load	File mapping, allocation and warm-up elapsed time
Prompt processing	Input-token count and processing rate
Time to first token	Request start to first meaningful output event
Decode	Generated tokens and decode time, with reasoning counted explicitly
End-to-end latency	Request to terminal success/error
Resources	Peak RAM/VRAM, host/device transfers, CPU/GPU use where observed
Reliability	Cancel, recovery, unload, timeout and failure behavior
Quality	Relevant task accuracy or numerical parity

A UI duration often includes prefill and reasoning; an engine’s tokens/second may report decode only. State which tokens and interval you counted. Preserve warm-up policy, repetitions and variability. Do not mix a cold run in one engine with a warm-cache run in another.

Minimal record

artifact: repository + revision + complete file hashes
engine: archive identity + actual reported version
machine: CPU/RAM/GPU/driver and available memory
load: exact context, placement, threads, batch, cache settings
workload: prompt/template/token counts/output bound
runs: warm-up policy, repetitions, raw timings and errors
result: quality check + latency/decode/resource summary

The current installed arithmetic result belongs in functionality evidence. A future optimization result needs its raw measurements, exact applied strategy and reproducible comparison before it changes a default or appears in marketing.