Give Qwen3.5 a prompt and every layer receives the same token stream, but every layer does not remember that stream in the same way. The repeating schedule is three Gated DeltaNet recurrent layers followed by one full-attention layer. One path compresses history into persistent state. The other preserves a searchable KV record of earlier tokens. C-Kernel-Engine (CKE) is an AI compiler/runtime that turns model circuits and weights into selected C kernels, memory plans, generated C, and a compiled executable path. For Qwen3.5, CKE must compile both memory systems, keep their state ownership separate, and reproduce the reference arithmetic closely enough that one floating-point rounding difference does not select a different token 40 steps later.

1. Start With The Concrete Model, Not The Family Name

“Qwen3.5” names a model family. This article examines the exact Qwen3.5-0.8B Q4_K_M path currently hardened in CKE v8. Its converted manifest contains 24 decoder layers, hidden width 1024, eight query heads, two KV heads, head dimension 256, and a full-attention interval of four. The recurrent branch uses state dimension 128, 16 groups, time-step rank 16, and inner width 2048.

CKE does not infer the schedule later from the model name. Conversion creates layer_kinds and a layer_execution_plan. Each layer explicitly owns one of four state policies: recurrent state or KV cache; no attention or full attention; DeltaNet or no recurrent core; no KV production or KV production. The executable policy is tested in the Qwen3.5 v8 policy suite and declared by the Qwen3.5 circuit.

2. Two Memory Contracts Share One Residual Stream

A normal dense transformer makes every attention layer retain K and V rows for earlier tokens. If there are L attention layers, context length T, H_kv KV heads, head width D, and b bytes per stored scalar, the cache contribution is approximately:

The factor two is for K and V. The important term is T: the cache grows as the conversation grows. Qwen3.5 does not eliminate this memory. It assigns full attention to six of 24 layers. Holding dimensions and storage precision fixed, the KV component is therefore roughly one quarter of an all-attention 24-layer decoder. That is not a claim that total model memory falls by four times; weights, activations, recurrent state, allocator padding, and runtime scratch remain.

A recurrent layer stores a convolution history and a matrix state instead. Abstractly, for G groups with key and value state widths D_k and D_v:

This state is updated for every token, but its shape does not contain context length T. It is compressed memory, not a retained row for every past token. Full attention is closer to photographic memory: earlier K/V rows remain individually addressable. DeltaNet state is closer to a learned working summary: new evidence modifies a fixed-size matrix. The metaphor describes the practical storage behavior; neither mechanism is human memory or consciousness.

Saying “Mamba or DeltaNet versus attention” misses this hybrid design. Qwen3.5 uses recurrence and attention together because they offer different memory costs and retrieval behavior. The decoder periodically buys full token-to-token interaction without paying that cache cost in every layer.

3. Kernel Circuit A: The Recurrent Layer

The recurrent layer is not one magical “linear attention” call. CKE’s circuit lowers it into an ordered chain of projections, local convolution, normalization, state update, gating, projection, residual addition, and MLP work:

1. RMSNorm: normalize the shared residual stream.
2. Five projections: produce recurrent Q/K/V, a feature gate, decay/alpha, and update strength/beta.
3. Split Q/K/V: interpret the packed recurrent projection by its declared dimensions.
4. Time-step gate: transform alpha and bias into the recurrent decay signal.
5. Short causal convolution: update a persistent local-history buffer and mix neighboring token features.
6. SiLU and split: apply the reference activation arithmetic, then recover convolved Q/K/V streams.
7. Q/K L2 normalization: normalize each recurrent head under the model’s exact denominator, clamp, and rounding semantics.
8. Gated DeltaNet: decay, read, correct, and query the persistent matrix state.
9. Recurrent norm + gate: RMS-normalize state output, apply SiLU to the feature gate, and multiply without changing reference operation boundaries.
10. Output projection + residual: return to width 1024 and merge into the residual stream.
11. SwiGLU MLP + residual: run the block’s common feed-forward tail.

The distinction survives compilation. Recurrent convolution state and DeltaNet matrix state have layer-scoped offsets and persist across decode calls. They are not temporary activation buffers that the memory planner may reuse. The implementation is visible in recurrent_state_kernels.c, recurrent_qk_norm_kernels.c, and deltanet_kernels.c.

4. Kernel Circuit B: The Full-Attention Layer

Every fourth layer follows a more recognizable transformer path, but Qwen3.5 still adds model-specific details:

1. RMSNorm the residual input.
2. Project a combined query-and-gate tensor, then split query from gate.
3. Project K and V separately.
4. Apply Q/K normalization and rotary position encoding.
5. Append K/V rows to that layer’s persistent FP16 cache.
6. Run causal grouped-query attention.
7. Evaluate sigmoid(gate) * attention_output.
8. Run output projection, residual addition, and the common SwiGLU MLP tail.

Only these layers own KV-cache rows. That is why a generic decoder loop that allocates cache for all 24 layers is semantically misleading and wasteful. Conversely, treating all layers as recurrent would remove the periodic full-attention behavior encoded by the trained weights. The circuit, generated call ABI, and memory plan must agree layer by layer.

5. What The DeltaNet Kernel Actually Computes

For one recurrent head, CKE’s AVX2 provider follows the pinned llama.cpp update. Q and K arrive normalized. Let S_t-1 be the previous matrix state, g_t the learned decay gate, and β_t the learned correction strength:

The symbols now expose the stages: d_t is decay, u_t is update strength, S_t is the decayed state, v̂_t is the value predicted by that state, and e_t is prediction error. First decay the old state. Then ask what value it predicts for key k_t. Compare that prediction with v_t, write an update-scaled correction with an outer product, and finally query the corrected state with q_t.

This is why recurrent state is more than a running average. The matrix stores key-to-value associations and updates them using an error-correction rule. It is also why arithmetic order matters: the dot-product tree, expf, sigmoid expression, FMA state update, normalization, and token update order all become part of the runtime’s numerical contract.

6. Why “The Output Looks Fine” Was Not Enough

What CKE Calls X-Ray

X-ray is CKE's numerical-attribution subsystem. It is not a CPU profiler, one kernel, or a graphical tensor viewer. A profiler answers where the runtime spent time. X-ray answers a different question: at which named edge in the model circuit did CKE first stop producing the same values as a trusted reference execution?

The implementation spans several parts of the v8 codebase because observing a compiled model requires cooperation between generated C, the runtime, the reference adapter, and the comparison logic:

1. Generated checkpoints: codegen_core_v8.py and codegen_prefill_v8.py place optional dump calls after semantic circuit edges such as normalization, Q/K/V projections, recurrent gates, attention output, MLP output, and logits.
2. C dump transport: ck_parity_dump.h writes a 128-byte record header plus tensor values. The header identifies the layer, operation, shape, dtype, and token. Layer and operation filters keep a bounded investigation from dumping the complete model. When CK_PARITY_DUMP is disabled, these calls compile to no-ops.
3. Reference capture: a model-specific adapter replays the same prompt and generated-token prefix through the comparison runtime. For this article, xray_text_recurrent_v8.py asks the pinned llama.cpp replay bridge for corresponding graph tensors.
4. Vocabulary translation: CKE checkpoint labels describe circuit edges, while llama.cpp labels describe graph nodes. The adapter maps names and occurrences explicitly instead of assuming that two tensors called Qcur represent the same point in the computation.
5. Layout canonicalization: tensors are reordered into one logical representation before comparison. For example, Qwen3.5 recurrent state is stored as [head,key,value] in CKE and [head,value,key] in the reference dump. Comparing the raw files without transposing one side would manufacture a false failure.
6. First-edge attribution: the analyzer compares every available value and reports shape status, differing-element count, maximum absolute difference, RMSE, the last exact checkpoint, and the first divergent checkpoint. A diagnostic materiality floor may prioritize where to investigate, but it does not hide smaller value differences from the report.

same prompt + same generated prefix
              |
       +------+------+
       |             |
  CKE instrumented   pinned reference capture
       |             |
  named circuit      named graph tensors
  edge tensors       and runtime metadata
       +------+------+
              |
   map names -> canonicalize axes -> compare values
              |
   last exact edge -> first divergent edge -> next test

Qwen3.5 forced X-ray itself to become schedule-aware. llama.cpp exposes prompt work as a batched prefix, while recurrent decode is a sequence of state transitions. The adapter therefore addresses tensors by logical token, distinguishes batched prefill from sequential decode, and only compares persistent recurrent state at points where both executions represent the same transition. A debugger that ignores execution schedule can confidently blame the wrong kernel.

CKE also has generic numerical-manifest tooling in xray_numerical_parity_v8.py, execution-state checks, vision adapters, schemas, and ranking reports. These pieces are still being hardened. A future dedicated article will trace one complete X-ray run from generated dump hook to first-divergence report and explain how new model families should register comparable semantic edges.

Short greetings produced coherent text while a longer C/Python/SQL prompt diverged from pinned llama.cpp. That is the dangerous class of runtime bug: the model appears functional, but a tiny early difference eventually changes token ranking. CKE did not solve this by increasing tolerance. It extended X-ray capture until the first wrong arithmetic boundary became observable.

One Algebraic Identity, Two Floating-Point Programs

CKE had used a sign-stable sigmoid branch for negative inputs. llama.cpp evaluated the direct expression unconditionally:

Over real numbers, the stable branch and direct expression are equivalent. In FP32 they need not round to the same bit pattern. X-ray showed exact attention immediately before the gate, a one-ULP gate difference, and later amplification when values crossed Q8 quantization thresholds and entered MLP projections. The fix was not “use the mathematically better sigmoid.” The fix was “implement the numerical contract selected for this reference-compatible circuit.”

This distinction matters beyond llama.cpp compatibility. A compiler/runtime must decide what it promises: mathematical tolerance, storage-level agreement, token agreement, or bit-exact reference execution. CKE uses different gates for different claims, but it does not silently replace one with another when a test is inconvenient.

7. X-Ray Had To Understand The Schedule Too

A diagnostic tool can produce false confidence if it observes the wrong execution model. Qwen3.5 prefill is a sequence of recurrent transitions, even when the reference graph exposes tensors in a batch. Decode is one new transition appended to persistent state. The X-ray adapter therefore had to:

• preserve batched-prefix versus sequential-decode modes explicitly;
• unpack reference tensors by logical token rather than compare one batch blob to one recurrent step;
• canonicalize recurrent state from reference [head,value,key] to CKE [head,key,value] before comparing values;
• capture missing RMSNorm, Q/K/V, gate, cache, attention, recurrent, and MLP boundaries;
• verify the generated runtime, engine, model, compiler, and reference provenance;
• stop at the first causal boundary instead of blaming the largest downstream mismatch.

The maintained diagnostic is xray_text_recurrent_v8.py. This is an important engineering result by itself: a hybrid runtime needs a hybrid-aware debugger. A dense-transformer tensor dumper cannot be assumed correct merely because every tensor has a familiar name.

8. Numerical Parity Is Not Model Quality

After the parity work, one local CKE chat run provided a useful reality check. Qwen3.5-0.8B produced polished headings, code fences, plausible explanations, and roughly 38 decode tokens per second on the recorded 20-thread CPU configuration. Surface fluency was good for a model of this size. Technical correctness was not.

Both programming answers stopped at the configured 512 generated-token limit, so their incomplete endings cannot be attributed entirely to reasoning failure. The substantive errors above appeared before truncation. Memory was also disabled, meaning each request was evaluated without retaining the earlier chat as context. This remains one informal transcript, not a coding benchmark or a representative quality score.

Parity asks whether CKE executed this checkpoint like the reference runtime. Evaluation asks whether the checkpoint's answer was correct, useful, safe, and appropriately concise. Passing the first question does not answer the second.

This is precisely why fluent output was only a smoke test during bring-up. CKE needs numerical parity gates for the runtime and separate task suites for model quality: compilable-code tests, unit tests, exact-answer datasets, long-context probes, tool-use evaluation, and human review. A runtime bug and a weak model answer can look similar at the chat surface, but they require completely different evidence and fixes.

9. What CKE Can Honestly Claim Today

On the evidence merged in PR #215, CKE v8 reproduces the tested Qwen3.5-0.8B Q4_K_M path against a pinned llama.cpp runtime for 64, 128, and 256 generated tokens on the promotion prompt. The gate oracle is exact, 68 resolver/X-ray/codegen checks pass, the numerical-contract suite passes with 34/34 FP16 attention cases, and staged v7/v8 regressions pass.

That is meaningful evidence. It is not universal certification. The boundary is:

• one Qwen3.5 checkpoint and quantization recipe;
• one promotion prompt and bounded generation lengths;
• a pinned llama.cpp commit and recorded runtime provenance;
• an AVX2 host and the exact providers selected there;
• inference, not complete end-to-end training parity;
• correctness evidence, not a claim that CKE is faster than llama.cpp.

AVX-512, other compilers, other Qwen3.5 sizes, other quantizations, long contexts, broader prompt suites, performance promotion, and full training remain separate gates. The useful result is not “Qwen3.5 solved.” It is that CKE now has an explicit hybrid circuit, layer-scoped state ownership, exact numerical providers, schedule-aware X-ray tooling, and a promotion record that another engineer can challenge.

The model emitted coherent code long before the runtime was exact. Fluency was a smoke test. The first wrong tensor was the engineering evidence.

Related Notes And Source Artifacts

• DeltaNet vs SSM: Two Fixed-Size Memory Systems Beyond Attention
• Templates Are Circuit Maps
• CKE Kernel Registry: How Exact C Functions And Numerical Contracts Connect
• Prefill vs Decode in CKE
• Qwen3.5 v8 circuit
• Gated DeltaNet C kernels
• Qwen3.5 recurrent parity certification commit

To inspect or contribute to CKE, use the contributor path or join the CKE Discord channel. Documentation, distribution compatibility, reproducible profiling, numerical fixtures, kernel implementation, compiler work, and model-family bring-up are all useful contributions.