Development plan
Written 2026-10-03 from the draft spec (docs/spec.md) and three research passes:
the Mojo 1.1 toolchain, the diffusion-LLM landscape, and the house style of the
author's other repositories. Where research contradicted the spec, the
correction is recorded here and the code follows the correction. The plan
predates the code: where the code later moved, the text below carries the
as-built wording, and Architecture maps what shipped.
Corrections to the spec
| Spec says | Reality (verified) | Consequence |
|---|---|---|
| Q4_K_M support, optionally Q8_0/F16 | A Q4_K_M GGUF of LLaDA-8B holds Q4_K (incl. token_embd), Q6_K (attn_v/ffn_down in half the layers, output) and F32 norms | v0.1 needs F32 + Q4_K + Q6_K |
mojoproject.toml | magic is gone; Mojo installs from PyPI | pin mojo==1.1.0, max-core==26.6.0 in pyproject.toml, commit uv.lock |
mojo test | removed; TestSuite, each test file built with mojo build and run as a binary; an empty suite exits 0 | CI pins the test count |
| llama.cpp as generation reference | its diffusion sampler skips CFG/alg_temp, noises one row, inverts entropy, samples at temp 0 | llama.cpp is the logit reference; token parity uses the LLaDA generate.py algorithm re-implemented over llama.cpp logits |
| diffuse-cpp as drop-in comparison | own diffuse GGUF schema, no tokenizer, unverified parity | benchmarked with its own file, on token IDs |
| RoPE | GGUF converter un-permutes Q/K: GGUF uses adjacent-pair (NORM) RoPE | NORM RoPE when reading GGUF |
UnsafePointer, fn, parallelize in std | 1.1 removed fn/alias; parallelize moved to max.algorithm | write 1.1 Mojo only |
Rows 3, 6 and 7 record assumptions from the research passes. docs/spec.md
uses fn, and names none of mojo test, UnsafePointer, parallelize or a
RoPE variant.
Model facts used (LLaDA-8B-Instruct, GGUF llada)
32 layers, d = 4096, 32 heads × 128 (MHA), FFN 12288 SwiGLU, RMSNorm ε = 1e-5,
RoPE θ = 500 000, vocab 126 464, mask 126 336, BOS 126 080, EOS 126 081,
EOT 126 348, non-causal, untied output. Tokenizer: byte-level BPE (gpt2),
pre-tokenizer bailingmoe.
Compute design
By default a denoising step runs the full transformer over the whole canvas (prompt + generation, L ≈ 100 to 160 tokens), so linear layers are GEMMs, not GEMVs. Cost per step ≈ 2 × 7.0·10⁹ × L FLOPs, plus 2 × 0.5·10⁹ per row whose logits are computed (LLaDA-8B): compute-bound, not bandwidth-bound.
- Activations are quantized per token to int8 with one f32 scale per 256 values (the Q8_K scheme), plus the sum of each group of 16 values as a float.
- At load, each Q4_K and Q6_K matrix is repacked so that a 256-bit register holds 4 consecutive weights of each of 8 rows. The two types have their own layouts and tile functions; The quantized GEMM has both.
- The inner product uses
vpdpbusd(AVX-VNNI) when the CPU has it, andpmaddubswpluspmaddwdotherwise. Onevpdpbusdmultiplies 8 rows by 4 activation bytes, and no horizontal sum is needed until the row ends. - The LM head runs only on the masked positions the sampler can commit in the
step: the current block for LLaDA (at most
--block-lengthrows, 32 by default) and every masked position of the canvas for Dream (at most--max-tokensrows), not all L rows. This is a diffusion-specific saving that a generic runtime does not take. - Threads split output rows into many small chunks so the 4 P-cores and 4 E-cores of a hybrid CPU balance.
Correctness ladder
- Unit: GGUF parse (synthetic files generated by
scripts/gen_fixtures.pyand committed undertests/fixtures/), Q4_K/Q6_K decode against hand-built blocks, quantized dot vs f32 dot, RMSNorm, RoPE, softmax, attention, scheduler transfer counts againstgenerate.py. - Tokenizer: golden token IDs produced by
llama-tokenizeon a fixed corpus. - Logits: a small C++ tool over libllama dumps logits for fixed token sequences; mojo-dllm is compared on cosine similarity, relative RMS error and argmax agreement. Correctness has the measured values.
- Generation: the same tool runs LLaDA's
generate.pygreedy algorithm over llama.cpp logits; the tokens are compared one by one, and a run that diverges reports the first position that differs.
Benchmark protocol
Same machine, same prompts, same thread count, release builds, model load
excluded from generation time and reported on its own. Configuration:
gen 128, block 32, steps 32 (4 tokens per step, 32 forward passes), plus
per-step latency. Three runs each, median reported, raw numbers in
bench/results/*.json. The README table is rendered from that JSON and a
gate fails when they disagree. No estimated numbers are published.
Repository gates
Sized to what this repo can break, copied from sysknife's reasoning:
| Gate | Guards against |
|---|---|
| staged secret + employer-name scan (pre-commit) | a key or the employer name in history |
mojo format then git diff --exit-code | unformatted code (there is no check mode) |
build with -Werror | deprecated APIs piling up across Mojo releases |
| tests with a pinned count | a suite that selects 0 tests and still exits 0 |
| shellcheck, yamllint, markdownlint | broken scripts and workflows |
| action pins verified against tags | a floating @v4 or a mislabelled SHA |
README numbers equal bench/results | hand-typed benchmark claims |
| mdBook build + internal link check | broken docs |
Development lists the full set, including the launcher tests, the GPU suite, the no-GPU check and two self-tests.
Local first: scripts/ci-local.sh runs everything; pre-commit runs the fast
set, pre-push the full set. GitHub Actions runs the same gates on push and PR,
except that it builds the GPU suite without running it, split into three jobs
(.github/workflows/ci.yml). .github/workflows/docs.yml deploys the book to
GitHub Pages on pushes to main that touch docs/, book.toml, theme/ or
assets/.
Milestones
| Deliverable | |
|---|---|
| M0 | repo, CI, logo, docs skeleton |
| M1 | mojo-dllm inspect model.gguf |
| M2 | Q4_K/Q6_K decode, quantized GEMM, tests, microbenchmark |
| M3 | RMSNorm, RoPE, attention, SwiGLU |
| M4 | LLaDA forward, logit parity with llama.cpp |
| M5 | tokenizer, scheduler, mojo-dllm run |
| M6 | threading and SIMD tuning, benchmark report vs llama.cpp and diffuse-cpp |
| M7 | NVIDIA backend (--device gpu), shipped in 0.2.0 |
| M8 | Dream-7B, shipped in 0.1.0 |
| later | block diffusion; inter-step caching beyond --dual-cache |
M0 to M6 and M8 shipped in 0.1.0. M7 shipped in 0.2.0.
The repository went public after M6, with parity documented and the license of
shipped binaries reviewed (Modular runtime libraries are not Apache; NOTICE
records the terms).