Strata: the Engine That Made 125B a Gaming-PC Hobby

Local AI Updated 2026-11-04 · 2 min read · #engine #moe #ram-offload #ssd

What it is

Strata is a local inference engine + guided launcher: you run strata launch, it prints a plan (nothing is downloaded until you say go), and starts the model. The trick is its memory layout: shard-1 experts load into RAM, a ~29 GB lookup table stays on NVMe, and your GPU only ever handles the active part per token. Windows + Linux, NVIDIA + AMD (plus Intel Arc and Apple Silicon paths), and everything reads pictures.

Getting started (3 steps)

Install, then strata launch --plan — the plan shows exactly what fits your machine before anything downloads.
Pick a pack by your RAM (from its docs, weights-in-RAM+VRAM): Q2_0 ≈ 37.6 GB, IQ2_XS ≈ 39.2, IQ3_XXS ≈ 47, IQ3_S ≈ 54.8 and “matches the full model on the published tests”. With 64 GB RAM, all fit; with 48, Q2_0/IQ2_XS do.
strata launch and go. --tune quick measures your box; draft layers (MTP ≈6 GB) come automatically.

What it runs (and measured speeds from its own docs)

Model	Sizes	Measured
Qwen3.8-Flash-Next 125B	above sizes; Unsloth 4–8-bit packs need 72–136 GB; NVFP4 ≈80 GB	RTX 5070 12GB: 94 tok/s (Q2_0, short chat), 76 at 128k ctx
DeepSeek-V4.1-Flash-1017 (748B)	ISTA 1.75–2.25 bpw, 239–274 GB packs	5 tok/s (RTX 5070, code); 6–9 (3060); 4× R9700 ≈31, 55–69 with DSpark drafter
GLM-5.3-Flash (321B)	supported via packs	see Maya for the rival path
Qwen3.6-35B-A3B	23–38 GB, Apache 2.0	59.7 tok/s on a 3060 with experts on CPU

Honest weaknesses

12 GB cards spend “most of each token waiting on the SSD” at 748B class — speeds are slow-typist unless your RAM covers the experts.
Answers can differ between runs on the same prompt (which experts sit on the card shifts moment to moment).
Docs-heavy project: the good numbers live in its docs, not in one-click defaults.

FAQ

Is this what you’re running on right now?

This very page’s site is static, but the model answering you in its family tree — the “Qwen3.8-Flash-Next Uncensored Q4_K_M (Strata)” builds are OrcaRouter-variant packs that need a manual packing step, not an installer menu item.

48 GB RAM with a 24 GB card?

Its low-RAM mode maps experts from files and keeps in RAM only what the card can’t hold — a 32 GB PC + 24 GB GPU runs Q2_0/IQ2_XS/Coder that way.

📌 Summarised from official docs and hands-on samples. Product features and pricing change often — official site wins.

← Newer: Qwen3.8-Flash-Next: the 125B That Made 100B-at-Home a Hobby