MoE, offload and expert residency

Total parameters still need a home

A mixture-of-experts model routes computation through selected experts. Its total parameters describe the complete weights; active parameters describe a workload-dependent subset used for a token. A small active count does not make the complete artifact disappear from RAM, storage or a future residency plan.

Different MoE architectures use different routing, shared experts, tensor layouts and state. A Qwen-family optimization cannot automatically be applied to a dense model, another MoE family or a recurrent hybrid. Inspect exact architecture and tensor names before choosing an execution strategy.

Ordinary layer offload

The current 0.0.1 baseline places selected transformer blocks on the GPU and leaves other work on the host. It is ordinary layer offload through the qualified llama.cpp backend. On a large MoE model, this can keep expert-heavy blocks on the CPU and become limited by CPU or memory bandwidth.

Successfully fitting a model this way establishes a functional path. It does not establish optimized expert caching, fast tokens or a throughput advantage over LM Studio.

More selective placement

One experiment is to keep shared/dense work on the GPU while retaining routed experts in host memory. Another is a bounded GPU expert cache with asynchronous transfers and predictive prefetch. These strategies differ in kernels, transfer cost, scheduling and supported tensor layouts. A flag that changes residency is not evidence that predictive paging has been implemented.

RegesARC’s optimization work must preserve every expert and the selected artifact. It must measure cache behavior, host/VRAM residency, transfer volume, first-token latency, decode throughput and correctness. If a strategy changes computation or quality, that tradeoff must be visible.

Hybrid state

RegesCore’s inspected architecture is qwen35moe, with hybrid/recurrent behavior. Attention KV is only one state component. Do not multiply a dense-transformer cache formula by every block and call it an exact allocation; do not omit recurrent state and call a small lower bound the whole footprint.

The Strata research guide describes the integration boundary. A native strategy should be promoted from experimental only when the same artifact produces validated output and a repeatable improvement on the declared hardware tuple.