C-Kernel-Engine, abbreviated CKE, is a compiler and runtime project that does not treat a model as one handwritten inference loop. It treats the model as a circuit of mathematical kernels that must be selected, connected, assigned memory, lowered into call-ready intermediate representations, and compiled into a standalone C runtime. The kernels are the gems of CKE, as they are in any machine-learning library. If a kernel computes the wrong values, every abstraction above it can be beautifully organized and still be wrong.

A kernel map is CKE's machine-readable representation of one concrete C kernel provider. It connects the kernel's mathematical operation to its supported dtypes, tensor shapes, storage boundaries, reductions, execution policy, C declaration, sources, tests, and ordered call ABI. The registry must therefore answer questions that a name such as attention, rmsnorm, or gemm cannot answer alone. Is this prefill or decode? Is the KV cache FP16 or FP32? Is the tensor head-major? Which reduction and rounding sequence must match the reference? Which pointers receive scratch buffers, runtime state, weights, outputs, dimensions, and constants?

1. The Four Documents People Commonly Collapse Together

CKE separates four responsibilities that are easy to confuse:

The word template survives in field names because the sidecar manifest embeds a hydrated template document, but v8 treats that architecture document as a circuit. The distinction is conceptual, not cosmetic. The circuit is the canonical model-semantic source consumed during IR1 generation. It can say that Qwen3-VL requires M-RoPE, Q/K normalization, an FP16 KV cache, head-major tensors, segmented multimodal prefill, and particular attention-reduction contracts. It should not say, “if model name equals Qwen3-VL, call this function.”

A kernel map faces the opposite direction. It does not understand the whole Qwen3-VL architecture. It advertises that one C provider can satisfy a precise combination of operation, phase, layout, storage, reduction, threading, and ABI requirements. Another model can reuse that provider if it requests the same semantics.

The circuit asks for meaning. The kernel map offers a measured implementation. The registry is the searchable catalog. The resolver proves the match.

2. How CKE Uses The Registry

The v8 path is methodical:

1. Conversion writes the model weights and a sidecar such as weights_manifest.json. The sidecar records tensor names, offsets, shapes, dtypes, quantization summary, model configuration, and an embedded or named circuit/template.
2. build_ir_v8.py hydrates the circuit from the sidecar and built-in circuit definition. The circuit supplies operation order, graph edges, state transitions, and required contracts; the sidecar supplies the actual model dimensions and weight storage.
3. During IR1/GraphIR generation, CKE expands each circuit operation and queries the kernel maps. Broad operations use their registered implementation identity; contract-governed operations are matched by the full requirement tuple.
4. The resolver validates both sides against strict schemas and requires the circuit's requested values to appear in the provider's advertised capabilities. Zero matches is a hard fault. Ambiguous matches are also hard faults.
5. IR1 records the selected kernel ID, function, numerical contract, execution contract, tensor dataflow, and weight bindings. This is where the abstract circuit first becomes a concrete kernel circuit.
6. The fusion pass may replace eligible sequences only through registry-declared patterns. The memory planner uses IR1 dataflow, tensor dimensions, lifetimes, and scratch requirements to assign reusable physical buffers and weight offsets.
7. LoweredIR preserves the resolved provider while converting logical inputs and outputs into concrete buffers, cache state, dimensions, and execution details. It may wire the call, but it may not choose different mathematics.
8. Call-ready IR consumes the ordered call_abi owned by that kernel map. It binds every C parameter to a weight pointer, scratch pointer, runtime cache, output buffer, dimension, position, or constant selector.
9. Code generation emits the stitched C calls, compiles the model runtime, and executes the same provider identities selected during IR1.

A compact representation of the flow is:

$$(\text{weights} + \text{sidecar} + \text{circuit})\xrightarrow{\text{IR1 + kernel-map resolution}}\text{resolved kernel circuit}\xrightarrow{\text{fusion + memory + lowering + call ABI}}\text{generated C}$$

This fail-closed policy matters. CKE explicitly forbids adding a silent fallback, relaxing a tolerance, ignoring an unknown contract field, or restoring model-specific provider selection inside later lowering or code generation. If IR1 cannot find a map that proves the circuit requirement, compilation should stop at that unresolved boundary rather than stitch plausible but semantically different code.

3. Circuit Requirement Versus Kernel Offer

The Qwen3-VL circuit currently asks for the following prefill-attention combination:

{
  "requires": {
    "execution.phase": "prefill",
    "execution.prefill_batching": "segmented_append",
    "tensor.q.dtype": "fp32",
    "tensor.kv.dtype": "fp16",
    "tensor.layout": "head_major",
    "numerics.attention_reduction": "f16_flash_auto_qtile64"
  },
  "validation": "validated"
}

This block belongs to the circuit because it expresses what correct Qwen3-VL prefill requires. segmented_append reflects the text-before, visual-prefix, text-after schedule. fp16 names the persistent KV storage boundary. head_major names the tensor layout. The reduction identifier names the arithmetic schedule that parity work established.

The matching kernel map advertises the same combination as capabilities it provides:

"provides": {
  "execution.phase": ["prefill"],
  "execution.prefill_batching": ["segmented_append"],
  "tensor.q.dtype": ["fp32"],
  "tensor.kv.dtype": ["fp16"],
  "tensor.layout": ["head_major"],
  "numerics.attention_reduction": ["f16_flash_auto_qtile64"]
}

The arrays allow a provider to advertise every capability it has actually validated. They are not invitations to claim broad compatibility. If an optimized implementation supports FP16 storage but only one reduction order, its map should say exactly that. A new combination needs a new or extended provider contract and evidence, not an optimistic fallback.

4. One Production Kernel Map, Section By Section

The complete map is named attention_forward_causal_head_major_gqa_prefill_append_f16cache_flash_auto_qtile64. The name is long because “attention” is not enough information. Here is what each section contributes.

Identity: id, op, variant, mode

{
  "id": "attention_forward_causal_head_major_gqa_prefill_append_f16cache_flash_auto_qtile64",
  "op": "attention",
  "variant": "causal_head_major_gqa_prefill_append_f16cache_flash_auto_qtile64",
  "mode": "prefill",
  "contract_schema_version": 1
}

op forms the broad family. variant distinguishes this provider from decode, full attention, sliding-window, shared-KV, FP32-cache, and other attention implementations. id is the stable identity that must survive resolution and every IR stage. mode prevents a prefill provider from being silently reused for decode merely because the pointers look compatible.

Capabilities: provides

This is the resolver-facing contract shown above. It turns hidden assumptions into queryable data. Before these combinations were explicit, later code could inspect a model family, cache dtype, or function name and select again. Recent CKE hardening removed those second decisions: once the circuit and map resolve, lowering consumes the answer.

Numerics: supported_reductions

"supported_reductions": {
  "f16_flash_auto_qtile64": {
    "status": "validated",
    "function": "attention_forward_causal_head_major_gqa_prefill_append_f16cache_contract",
    "explicit_selector": true,
    "selector": "CK_ATTN_REDUCTION_F16_FLASH_AUTO_QTILE64",
    "notes": "Q<64 uses FP16 online single-range arithmetic; Q>=64 uses 64x64 tiled FP32 arithmetic."
  }
}

“FP16 attention” is still underspecified. A complete reduction contract can distinguish Q and K compute precision, score accumulation, softmax statistics, value accumulation, partial storage, partitioning, and partial-merge order. These decisions change floating-point results even when the high-level equation remains softmax(QK^T)V.

The selector is explicit because a C function may support more than one validated arithmetic strategy. The compiler does not infer one from tensor shape or ISA. It passes the named selector chosen by the resolved contract. Shape-dependent behavior can exist inside that named public provider only when every internal route is part of the same validated contract.

Execution: implementation

"implementation": {
  "isa_dispatch": "runtime",
  "threading": {
    "runtime": "serial",
    "work_partition": ["serial"],
    "dispatch": ["inline"],
    "reduction_order_effect": "contract_defined"
  }
}

Execution policy is separate from mathematical identity, but it can affect numerical identity. If workers split a reduction differently, the merge order and partial dtype must belong to the contract. This field prevents “same kernel, more threads” from being accepted automatically when the threading schedule changes arithmetic.

Tensor Interface: inputs, outputs, dims, params

"inputs": [
  {"name": "q", "dtype": "fp32", "shape": ["num_heads", "q_tokens", "head_dim"]},
  {"name": "k_cache", "dtype": "fp16", "shape": ["num_kv_heads", "cache_capacity", "head_dim"]},
  {"name": "v_cache", "dtype": "fp16", "shape": ["num_kv_heads", "cache_capacity", "head_dim"]}
],
"outputs": [
  {"name": "output", "dtype": "fp32", "shape": ["num_heads", "q_tokens", "head_dim"]}
],
"dims": ["num_heads", "num_kv_heads", "q_tokens", "past_tokens",
         "cache_capacity", "head_dim", "aligned_head_dim"],
"params": [
  {"name": "reduction", "dtype": "ck_attention_reduction_t"}
]

These fields expose the logical tensor and scalar interface. They let validation catch an FP32/FP16 storage mismatch, incorrect cache capacity, missing past-token position, or absent reduction selector before C compilation. They also make the map inspectable by tooling rather than requiring a reader to reverse-engineer a declaration.

Implementation Ownership: impl

"impl": {
  "function": "attention_forward_causal_head_major_gqa_prefill_append_f16cache_contract",
  "c_declaration": "ck_attention_status_t attention_forward_...(const float *q, const uint16_t *k_cache, ...);",
  "sources": ["src/kernels/attention_kernels.c"]
}

This section binds the semantic provider to a public C symbol, declaration, and source file. The stable kernel ID and C function are both validated. A renamed or substituted symbol cannot quietly preserve one identity while changing the other.

Call Construction: call_abi

"call_abi": {
  "version": 1,
  "params": [
    {"name": "q", "source": "scratch:q_scratch", "cast": "const float*"},
    {"name": "k_cache", "source": "runtime:kv_cache_k_layer_f16", "cast": "const uint16_t*"},
    {"name": "v_cache", "source": "runtime:kv_cache_v_layer_f16", "cast": "const uint16_t*"},
    {"name": "output", "source": "output:out", "cast": "float*"},
    {"name": "q_tokens", "source": "dim:seq_len"},
    {"name": "past_tokens", "source": "runtime:prefill_start_pos"},
    {"name": "reduction", "source": "const:CK_ATTN_REDUCTION_F16_FLASH_AUTO_QTILE64"}
  ]
}

This is one of the most important changes in the recent compiler work. Numerical resolution previously could choose the correct provider while a separate, function-keyed legacy table still owned argument order and sources. The result could fail late or bind the wrong runtime state. Kernel maps now own the ordered call ABI for governed providers. Call IR asks the selected map where each argument comes from: scratch storage, runtime cache, output buffer, model dimension, runtime position, or constant selector.

5. Why The Maps Became This Detailed

The detail was not designed in one pass. Git history shows it accumulating as real model work exposed every place where a broad kernel name hid an incompatible implementation.

Era One: Structural Kernel Descriptions

Commit 0ca399a7 added the first v8 vision-lowering maps on March 27, 2026. They covered generic operators such as tiled position embeddings, LayerNorm, M-RoPE, spatial merge, packed QKV splitting, feature slicing, and FP16 GEMM. Commit b97a28e2 then landed the broad v8 registry during Qwen3-VL multimodal bring-up on April 7. GitHub does not associate either early commit with a pull request, so they are retained here as the archaeological baseline rather than assigned an invented PR number.

An early attention map was already useful and detailed. It described:

• kernel identity, operation, and variant;
• weight, activation, and output quantization;
• inputs, runtime buffers, outputs, scratch, dimensions, and parameters;
• supported parallelization and preferred prefill/decode strategies;
• alignment constraints;
• the public C function, declaration, source files, and ISA variants;
• unit, benchmark, and parity tests with a broad tolerance.

This was enough to represent the kernel structurally and help IR1 choose functions, allocate scratch, and wire dimensions. It was not enough to prove numerical identity. For example, the early FP16-KV attention map said that K/V inputs were rounded through FP16 and compared against llama.cpp with a 1e-4 tolerance. It did not fully name how scores were accumulated, how online-softmax statistics were updated, how workers partitioned KV rows, how partials were stored and merged, or whether shape changed the arithmetic route.

The original maps answered “what buffers and function does this kernel use?” The hardened maps must also answer “which exact arithmetic and execution contract does this function promise?”

Era Two: Reduction Became A Model-Correctness Problem

PR #131, Make numerical contracts explicit (99bc42dd) records the turning point directly: circuits described operator order, while precision-sensitive reduction semantics were still inferred from runtime flags, cache dtype, token thresholds, and thread count. Correct leaf kernels therefore did not guarantee correct stitched behavior. The PR renamed v8 templates to circuits, added complete attention-reduction definitions, let circuits declare required_contracts, made executable kernel maps advertise provides and supported_reductions, and required exactly one provider. Its focused contract gate passed 10/10 and the pinned split-KV llama.cpp oracle passed 14/14.

PR #132, Make kernel contracts authoritative (a02c6f98) then closed the loophole where lowering could verify or infer a different route after selection. It introduced strict Draft 2020-12 schemas, hard-failed unknown fields and semantic environment overrides, resolved attention as part of IR1 construction across the supported model families, and carried required_contract, resolved_contract, and resolved_execution through GraphIR, LoweredIR, and call-ready IR. The contract gate increased to 18/18 while the same 14/14 external attention oracle remained green.

A modern attention-reduction definition can separately identify:

• Q and K compute precision;
• score accumulator precision;
• softmax maximum and denominator precision;
• V compute and output accumulator precision;
• partial-result storage dtype;
• single-range, worker-chunk, or query/key-tile partitioning;
• partial merge dtype and merge order;
• shape thresholds that select validated internal schedules.

These are not decorative details. Floating-point addition is not associative. Changing the number of partial sums, their dtype, or their merge order changes bits. Those small changes can cross a Q8 quantization boundary, perturb attention history, and eventually change an autoregressive token.

Era Three: Compiler Ownership Moved Into The Maps

Era Four: Real Model Parity Kept Expanding The Contract

This history explains why “just add a reduction” is the wrong response to a parity failure. A reduction is not a performance label. It is a named combination of compute dtype, accumulator dtype, storage boundaries, partition shape, merge order, and sometimes compiler-visible operations. Registering a new name without a scalar or external oracle, public-route test, and stitched model evidence would merely give an unvalidated path a respectable label.

6. The Registry Is Generated, But The Maps Are Authoritative

CKE currently aggregates the individual maps into KERNEL_REGISTRY.json. The generated file provides one searchable catalog and summary by operation. It is useful for the compiler, IR Hub, capability reporting, and validation, but it must not become a second hand-edited source of truth.

The source map remains the reviewable unit. A change to a provider should update its map, schemas, resolver behavior when necessary, and evidence. Registry generation then preserves source provenance such as the originating map file. Missing source maps behind a generated entry are a contract defect, not harmless generated-file drift.

7. The Q=63 ULP Failure Is An Application, Not The Definition

With the architecture established, the recent short-prefill incident becomes easier to place. In the attention map, q_tokens is the number of query rows. The named f16_flash_auto_qtile64 contract uses one validated arithmetic path below 64 rows and another at or above 64. Therefore Q=63 is the final boundary case on the short path.

PR #190 records the complete incident. Hosted AVX2 testing found that this case differed from the pinned llama.cpp production oracle by one FP32 ULP, one adjacent representable floating-point step at the affected magnitude. The online-softmax denominator update left multiply-add contraction implicit. Making the intended fused operation explicit with fmaf restored exact parity without changing the provider, route, storage, threading, or tolerance.

That incident validates the registry design because the surrounding identity was already fixed. Engineers could ask whether the selected provider, selector, shape route, thread schedule, and runtime artifact were unchanged, then isolate one arithmetic operation. Without those contracts, the same mismatch could be “fixed” by swapping kernels, changing dispatch, or loosening tolerance without knowing which behavior production actually used.

8. What The Registry Does Not Yet Magically Solve

• A detailed map does not prove its claims; executable leaf, public-route, stitched, and end-to-end evidence must support them.
• A registry does not make all kernels interchangeable; it makes incompatibility explicit.
• Runtime ISA dispatch remains safe only when every internal implementation preserves the advertised contract.
• Some older compatibility paths still use legacy bindings and measured specialization debt remains in the DSL and bridge.
• Performance ranking is not semantic resolution. The compiler must not guess a provider from benchmark data when the contract is unresolved.

9. Read The Production Sources

Read the Qwen3-VL circuit requirement, the matching production attention map, the v8 fail-closed contract policy, and the published numerical-contract documentation. The recent implementation history is visible in the merged CKE pull requests, particularly the codegen-capability, call-ABI, circuit-authority, attention-reduction, packed-quantization, runtime-provenance, and native-ISA parity changes.

Related ShivasNotes Research

This article sits between CKE's model-description layer and its executable numerical layer. The following earlier notes provide the surrounding architecture:

• What Is The C-Kernel-Engine? introduces the compiler/runtime thesis and explains why CKE generates a standalone model-specific C runtime.
• Templates Are Circuit Maps explains the architecture document that enters IR1 and requests the semantics resolved by the kernel registry.
• The CKE v8 IR And Code-Generation Pipeline follows operations through GraphIR, memory planning, lowering, Call IR, and generated C.
• K-Quants Deep Dive provides the block-format and mixed-dot-product background behind the Q4/Q6 × Q8 production-provider contracts discussed here.
• Threadpools And Memory Pools explains why thread partitioning, scratch ownership, and persistent runtime state cannot be separated from a CPU kernel's execution contract.
• Nemotron Kernels From Inference To Backpropagation demonstrates the kernel-by-kernel decomposition across inference, training-forward, and backward execution.
• Gemma4-VL In CKE shows how a multimodal model becomes a circuit whose vision and decoder operations require explicit kernel contracts.

CKE development is public and evidence-driven. If you want to test another CPU, improve a kernel map, or investigate numerical behavior, join the CKE Discord community, explore the C-Kernel-Engine repository, or support the independent research.