C-Kernel-Engine (CKE) is a Linux CPU runtime that turns explicit model and kernel contracts into generated C. Its v8 model matrix includes Qwen, Gemma, Nemotron, NaanBeige, and other model-family lanes at different maturity levels. GLM4 is one of those lanes, and this post follows what its bring-up required at the circuit, conversion, generated-dataflow, and runtime-smoke levels.

GLM4 is interesting because it is architecturally almost the same as Qwen/Llama-style models — dense decoder, RMSNorm, and a SWiGLU FFN — but has enough differences that a generic "llama-like" template can silently produce wrong output. For the GLM-4-9B-0414 lane, those differences include pairwise partial RoPE, QKV bias, post-attention and post-FFN norms, and the conditional [gMASK]\n prompting contract.

GLM4 vs The Field: What Actually Differs

Each bold cell is a contract CKE must represent rather than infer from a broad family resemblance. Pairwise RoPE rotates adjacent dimensions instead of CKE's split-half Qwen layout. Partial rotary leaves part of each head unrotated. Post-attention and post-FFN norms insert additional operations before their residual additions. A mismatch can remain numerically finite while still changing every downstream layer.

This table focuses on GLM4 against three well-known families. For the complete nine-family comparison — including hybrid recurrent (Qwen3.5), Mamba2 (Nemotron-H), and latent-attention (Kimi/MLA) architectures — see the full model-kernel matrix near the end.

// ln1_gamma: [embed_dim]
y = x * rsqrt(mean(x²) + eps)
y = y * gamma

The CKE Bring-Up Pipeline

Every new family goes through the same categories of evidence, but they are not automatically all green. For GLM4, the circuit and conversion contracts are executable tests; the hidden-state and logit comparators are diagnostic tools; and the current runtime evidence is a coherent Q4_K_M smoke. Applying split-half RoPE to this pairwise checkpoint is one example of a defect that can preserve finite numbers while corrupting the model's semantics.

What changed in the repository

The second change is especially important. A circuit describes the semantic producer, such as normalized main_stream. A selected quantized kernel may physically consume main_stream_q8, a derived packed view. Encoding only one side either loses model meaning or lies about the ABI. The GLM4 tests now guard both views instead of treating them as interchangeable.

version/v8/scripts/convert_safetensors_to_bump_v8.py \
  --checkpoint ./glm4-9b-hf \
  --arch glm4 --dtype bf16 \
  --output weights.bump \
  --config-out config.json --manifest-out manifest.json

Reproduce This Yourself

The whole point of bring-up infrastructure is that it is runnable, not aspirational. Start with the low-memory contract tests. On a host with enough RAM for the model, use the documented GGUF smoke command. The comparison tools require a matching safetensors checkpoint and a generated CKE model directory; they are not automatically executed by the smoke command.

# GLM4 circuit/dataflow contract: no 9B checkpoint required
.venv/bin/python -m unittest tests.test_v8_glm4_template

# Optional conversion coverage requires the test dependencies
.venv/bin/python -m pytest tests/test_v8_safetensors_to_bump.py -q

# Current documented Q4_K_M runtime smoke
version/v8/scripts/cks-v8-run run \
  hf://unsloth/GLM-4-9B-0414-GGUF/GLM-4-9B-0414-Q4_K_M.gguf \
  --context-len 1024 --force-compile --force-convert \
  --chat-template=glm4 \
  --prompt 'Give me a detailed example of C, Python and SQL code.' \
  --max-tokens 256 --temperature 0.0 --generate-visualizer

# Optional diagnostic: compare exported hidden boundaries
version/v8/scripts/compare_glm4_torch_ck_hidden_v8.py \
  --safetensors ./GLM-4-9B-0414 \
  --ck-hidden-dir ./ck_hidden_dump \
  --layers all --fail-cos 0.995 --fail-max 2.0

# Optional diagnostic: compare final logits
version/v8/scripts/compare_glm4_torch_ck_v8.py \
  --safetensors ./GLM-4-9B-0414 \
  --ck-model-dir ./generated_glm4_model \
  --top-k 16 --min-topk-overlap 0.50

The GGUF command above is copied from the current CKE model-kernel matrix. A different checkpoint can change tensor names, dimensions, chat metadata, or quantization paths, so passing this smoke does not automatically certify another GLM-family release.

The Tensor Map: HF → CKE

GLM4 uses HuggingFace naming conventions for its safetensors. CKE uses its own flat naming. The declarative mapping (safetensors_ck_map.json) handles this translation, including synthesized zero biases where HF has none but CKE's template contract expects them:

// safetensors → CKE tensor mapping (GLM4)
model.embed_tokens.weight        → token_emb
model.layers.N.input_layernorm   → layer.N.ln1_gamma
model.layers.N.self_attn.q_proj  → layer.N.wq
model.layers.N.self_attn.q_proj.bias → layer.N.bq
model.layers.N.self_attn.k_proj  → layer.N.wk
model.layers.N.self_attn.k_proj.bias → layer.N.bk
model.layers.N.self_attn.v_proj  → layer.N.wv
model.layers.N.self_attn.v_proj.bias → layer.N.bv
model.layers.N.self_attn.o_proj  → layer.N.wo
model.layers.N.post_attention_layernorm → layer.N.post_attention_norm  ★
model.layers.N.pre_feedforward_layernorm → layer.N.ln2_gamma
model.layers.N.mlp.gate_proj + up_proj → layer.N.w1  (concat)
model.layers.N.mlp.down_proj    → layer.N.w2
model.layers.N.post_feedforward_layernorm → layer.N.post_ffn_norm  ★
model.norm.weight                → final_ln_weight

The ★ tensors are GLM4-specific. If the mapper does not handle them, CKE would compile and run — but with wrong residual scaling at every layer.

Pairwise RoPE: Why It Matters

CKE's Qwen3 RoPE path uses a split-half layout: dimension i is paired with i + d/2. GLM4 uses a pairwise layout: adjacent pairs (0,1), (2,3), (4,5), ... are rotated together. Both are valid rotary layouts, but they are not interchangeable.

Additionally, GLM4 applies RoPE to only 50% of head dimensions (partial_rotary_factor = 0.5). The first half gets position information; the second half is position-agnostic. This is declared in the circuit JSON as partial_rotary: true and the converter reads partial_rotary_factor from the model config.

// Split-half RoPE (Qwen3):
// Pair dim[i] with dim[i + d/2]
for i in 0..d/2:
    (q[i], q[i+d/2]) = rotate(q[i], q[i+d/2], theta*pos)

// Pairwise RoPE (GLM4):
// Pair dim[2i] with dim[2i+1]
for i in 0..d/2:
    (q[2i], q[2i+1]) = rotate(q[2i], q[2i+1], theta*pos)

// Partial rotary (GLM4):
rotary_dim = head_dim // 2
q[:rotary_dim] = pairwise_rope(q[:rotary_dim], pos)
q[rotary_dim:] = q[rotary_dim:]  // no rotation

This is the GLM4-specific view. To see how pairwise stacks up against split-half (Qwen/Gemma3), split-direct per-layer (Gemma4), and MLA partial-concat (Kimi) — with an interactive dimension-pairing diagram for each — jump to RoPE Layouts below.

The Hidden-State Comparator

CKE's compare_glm4_torch_ck_hidden_v8.py hooks into every layer's intermediate tensors in the HuggingFace model and compares them against CKE's exported hidden states. This is how you find the first wrong operation:

If q_proj agrees but post_attn_norm does not, the first exported failure lies after the Q projection and at or before that later boundary. The comparator narrows the search interval; it does not by itself prove which kernel inside that interval is wrong.

The GLM4 Chat Contract

GLM4 has a distinctive prompting contract. When the tokenizer's add_bos_token is false (common in HuggingFace configs), CKE forces the prefix [gMASK]\n. The chat template uses role markers like <|user|>\n and <|assistant|>\n. Stop markers include <|user|> and <|observation|>.

// GLM4 chat format
[gMASK]
<|system|>
You are a helpful assistant.

<|user|>
What is pairwise RoPE?

<|assistant|>
{generated response}

// Stop markers: <|user|>, <|observation|>
// Min response tokens: 8

Zooming Out: GLM4 In The Full CKE Model Family

Everything above was GLM4 in isolation — its block, its RoPE, its parity gates. But the real payoff of the circuit-first approach only becomes visible when you step back and look at GLM4 next to every other family CKE supports. GLM4 bring-up validates the claim that the template/circuit system is genuinely family-agnostic: the same pipeline — circuit JSON → build_ir → codegen → compile → parity — handles GLM4's quirks (pairwise RoPE, partial rotary, extra norms, bias tensors) without any special-case code in the engine. The differences are declarative. They live in glm4.json and the tensor map, not in hand-written kernel branches.

The rest of this post is the wide-angle view: how GLM4's op sequence compares to Qwen, Gemma, Llama, and the more exotic hybrid and latent-attention architectures — and exactly which kernel decisions separate them. The three interactive diagrams below are built directly from the circuit JSON files in v8/circuits/.

CKE Model Architecture Comparison: Op Sequences Side By Side

Every model in CKE is defined by a circuit JSON file that declares its exact op sequence. This is not metadata — it is the source of truth the compiler uses to generate C code. Here is what each family's decoder block actually looks like, read directly from the v8/circuits/ directory:

Kernel Divergence: What Breaks If You Use The Wrong One

The kernel is where the contract becomes computation. Each model family requires specific kernel variants — use the wrong one and output degrades silently. Here is the divergence map across CKE's supported families:

RoPE Layouts: Why The Same Math Produces Wrong Results

Rotary Position Embedding (RoPE) is the most dangerous divergence point — it affects every token at every layer, but the difference is invisible without parity testing. Three models, three layouts:

// Pairwise: rotate adjacent pairs
for (int i = 0; i < head_dim; i += 2) {
  float x0 = q[i], x1 = q[i+1];
  float cos_i = cos_cache[i/2];
  float sin_i = sin_cache[i/2];
  q[i]   = x0 * cos_i - x1 * sin_i;
  q[i+1] = x0 * sin_i + x1 * cos_i;
}

Training Context: What Matters To The Runtime

The ChatGLM technical report explains the broader GLM family and its tool-oriented post-training, while the official GLM-4 repository records released checkpoints and usage conventions. Those sources are useful context, but CKE cannot infer an executable runtime contract from a training narrative alone.

CKE acts on checkpoint-visible facts: tensor names and shapes, tokenizer metadata, special-token policy, rotary dimensions, normalization tensors, attention geometry, and the declared chat format. Claims about why a model designer chose partial RoPE or post-block normalization are separate from proving how the released weights must execute.

Full CKE Model-Kernel Matrix

This table extends the official CKE model-kernel-matrix with architectural detail relevant to bring-up decisions. Every cell represents a contract decision that the template compiler must get right:

The pattern is clear: modern model diversity is not cosmetic. Each family makes different engineering decisions about how to encode position, gate information, mix sequences, and compress state. CKE handles this by making every decision explicit in the circuit JSON — the compiler reads the contract and selects the right kernel. No heuristics, no "llama-like" fallbacks.

Practical Takeaway

Model support is not a checkbox. It is a measurable pipeline: convert → template match → IR lower → compile → hidden parity → logit parity → coherence. GLM4 adds three new contract requirements to CKE's vocabulary (pairwise RoPE, partial rotary, post-block norms) — and each one would have caused silent degradation if the parity infrastructure did not catch it.

The kernel divergence map above shows why this matters at scale: as CKE grows to support 10+ model families, each with different RoPE layouts, activation functions, attention mechanisms, and state management contracts, the circuit-first approach prevents the combinatorial explosion that plagues "if-else" runtimes. The template is the contract. The contract is the compiler input. The compiler produces the right kernel calls or refuses to compile.

To continue from the model-family contract into code generation, read Templates Are Circuit Maps, then CKE's Kernel Registry. Together they explain how a circuit such as glm4.json becomes typed kernel calls rather than a pile of model-name conditionals.