Load Lab

The loading contract

Load Lab joins three decisions: which exact file, which runtime backend, and which memory/context settings. It does not silently quantize, remove experts or replace a large model with another artifact to make a load pass.

Control	Current behavior
Model	A discovered local GGUF language-model artifact
Backend	CUDA or CPU
Context	Exact integer, 128–262,144 and no greater than the model’s known ceiling
GPU layers	Explicit transformer-block count; zero uses host placement
All layers	-1 resolves to all model blocks; this is not an intelligent memory-fit optimizer

The planner requires a usable block count to estimate GPU placement. Selecting all layers on a model larger than GPU memory can be blocked. CPU mode uses zero GPU layers. A high host-side validation ceiling does not mean a model contains that many blocks; the artifact’s actual geometry is the relevant bound.

Experimental placement in the development branch

The current branch adds an explicit GPU + RAM experts (MoE) option and CPU threads input. This strategy completed actual installed-application load, generation, Stop, recovery, unload and saved-history qualification on the exact reference artifact. The earlier proof separately exercised ordinary layer offload. Existing defaults migrate to ordinary placement and zero explicit threads, without silently opting users in.

The MoE option requires CUDA and a complete, non-overlapping routed-expert tensor inventory. It fixes GPU layers to All while routed experts compute on CPU/RAM. That does not mean all weights reside on GPU. Host admission conservatively retains a complete-file budget; current free memory already excludes a loaded model’s allocation, so unload explicitly before recalculating a new placement.

Use MoE placement selects the strategy and up to eight CPU threads while retaining your exact context. Threads0 uses the backend default; a positive integer is bounded by logical cores and the host limit. Load/reload applies the settings; editing the controls alone does not persist them. Dense or unclassified layouts remain ineligible.

This is llama.cpp CPU-routed-expert placement, not Strata or predictive GPU paging. Read the exploratory observation before interpreting the profile as a performance guarantee.

Experimental Strata engine

The current preview includes a separate engine selector and an explicit GPU expert-cache budget. This adapter completed installed functional qualification on the exact reference profile; it is not the Lemonade MoE option. Its initial target is one exact RegesCore 35B MXFP4 file, with host-side SHA-256 verification, CUDA, fixed non-routed GPU layers and a 256–8,192 context bound. The input is never shortened when switching engines.

Strata cache sizes are 512–4,096 MiB in 256 MiB steps. Zero CPU threads resolves to up to eight total participants; explicit counts start at two. See the exact experimental profile before selecting it. The header reports the active runtime, independently from the draft in this selector. In System, the Strata maximum-output limit is 1–4,096 tokens per response; the Lemonade range remains 1–32,768. Switching engines preserves an existing value and reports a violation instead of silently reducing it.

Before you press Load

Review Memory & placement and the actual hardware snapshot. It estimates selected GPU weights, remaining host weights and an attention KV lower bound. Runtime workspaces, recurrent state, nonuniform layers and current competing allocations can invalidate a coarse estimate.

If the Lemonade backend is not ready, install it explicitly. Strata uses a separate private bundled runtime and has no automatic download button. Backend installation and model loading are distinct operations. Installation can require network access and disk space; importing a model does not trigger an automatic backend download.

While loading

The host serializes mutating operations. Wait for loading to settle before refreshing, installing, unloading or starting generation. A status event can arrive before the host finishes its final hardware refresh; the working indicator is part of the operation’s completion state.

The qualified load path disables automatic context fitting and requests the chosen integer. Check the runtime logs for the applied context and GPU-layer arguments. If allocation fails, preserve the original error, change one constraint deliberately, and retry. The current app does not advertise an effective Cancel load button because that upstream cancellation path has not been qualified.

After loading

Model loaded enables Chat. Unload current model releases the owned model instance; it does not delete the file. Reloading with changed settings is another allocation operation and must pass admission against current memory. See exact context and memory estimates for the assumptions behind the gate.