Today I want to write about one of the least glamorous and most important decisions inside C-Kernel-Engine (CKE) — my CPU-first AI compiler that turns a model into explicit, generated C kernels. The decision is this: when the compiler reaches an operation like a GEMM, an attention, or a KV-cache write, which concrete C function should it call? That sounds trivial. It is not. Getting it wrong does not crash the program. It produces a model that loads, talks fluently, and is quietly, numerically wrong.

I wrote earlier about what a kernel map is — the machine-readable contract that describes one real C provider. This post is the sequel: not what a map is, but how the v8 resolver actually picks one map out of 285, and why that logic lives in data and a mechanical algorithm instead of hand-written conditionals.

Why CKE Even Has Kernel Maps

It is worth stepping back and asking the blunt question: why does a compiler need a whole registry of JSON files to describe its own kernels? Most inference engines do not have anything like this. The reason comes from what CKE is trying to be. CKE does not ship one hand-written inference loop per model. It treats a model as a circuit of mathematical operations and generates a standalone C runtime for it. That means the compiler — not a human — has to decide which concrete C function fulfils each operation, and it has to make that decision correctly across every model, quantization format, and CPU it supports.

Without kernel maps, that knowledge has to live somewhere — and the only other place it can live is in code, as conditionals scattered through the generator. That is exactly the arrangement CKE is built to avoid, because a mistake there is invisible. A kernel map moves that knowledge out of code and into inspectable data, where it can be schema-checked, audited, diffed in a pull request, and matched against a numerical contract. The map is how CKE writes down, in a form a machine can verify, the promise that this C function computes exactly this arithmetic.

So kernel maps are not an accessory to CKE — they are the mechanism that lets the compiler own kernel selection at all. Take them away and the compiler either guesses from the model name (unsafe) or asks a human to wire every op by hand (does not scale). Kernel maps are what make automatic, correct, auditable selection possible.

Why This Cannot Be An If-Statement

Start with the size of the problem. One CKE runtime has to serve a product space:

model families   ×  quant formats  ×  ISA variants   ×  phases
Qwen2 / Qwen3    ×  Q4_K / Q5_1    ×  AVX2 / VNNI     ×  prefill
Gemma3 / GLM4    ×  Q8_0 / BF16    ×  AVX-512 / AMX   ×  decode
Nemotron / MoE   ×  FP16 / FP32    ×  scalar          ×  training / backward

Every cell in that space may need a different kernel. If provider choice is a Python conditional — if activation_is_q8 and cpu_has_vnni: call gemm_vnni() — then the failure mode is not a crash you can see. It is a branch that fires in a case nobody tested, selects a kernel with a slightly different reduction order, and returns numbers that pass every leaf unit test while corrupting the stitched model. On CPUs, where I am matching bit-for-bit numerical contracts, that is the single most dangerous class of bug.

A wrong if does not throw. It computes the wrong thing confidently. Fail-closed selection converts that class of silent-numerics bug into a compile-time error with a recorded reason.

Three Separated Jobs

The v8 design splits one decision into three layers that are deliberately not allowed to know each other's business. What math is required is separate from which provider runs is separate from how the op is physically executed.

• Circuits declare the logical. version/v8/circuits/*.json (like qwen3vl.json) list op instances, named port edges, and required_numerical_contracts. For a hardened op, a circuit must not name a physical provider ID. It states requirements, not implementations.
• Kernel maps own the physical. The 285 maps in version/v8/kernel_maps/*.json own the operation interface, the numerical-contract identity, port dtype/layout/stride/storage, phase support, persistent-state and alias semantics, the C call ABI, lifecycle status, and priority.
• The DSL resolves mechanically. build_ir_v8.py filters by contract, then by phase, dtype, shape, layout, ISA, and alias compatibility, ranks by lifecycle then priority, and fails closed on ambiguity. No model-name checks are allowed in the resolver.

What A Kernel Map Communicates

Before we watch the resolver run, it helps to know exactly what it is reading. A kernel map is not a copy of the C source — it is the machine-readable advertisement of what one C provider can do and under what promises. Every map communicates answers to questions a bare function name like gemm or attention can never answer on its own:

Put simply, a kernel map communicates a single sentence in a form a machine can check: "this C function computes this operation, with these types and this layout, in these phases, on these CPUs, called exactly this way, and here is the reference that proves it is correct." The resolver's whole job is to find the one map whose sentence matches what the circuit asked for.

How To Actually Read A Kernel Map

Enough abstraction — let me put a real map on the table and read it the way the compiler does. This is memcpy.json, the simplest production map in the tree. It implements residual_save (saving a residual is, at the bytes level, just a copy). I have trimmed a couple of fields with …, but nothing is invented:

{
  "id": "memcpy",
  "op": "residual_save",
  "operation_interface": "residual_save.memcpy_copy.v1",
  "selection": {"status": "production", "priority": 100,
                "equivalence_group": "residual_save.memcpy_copy.v1",
                "phases": ["prefill", "decode"]},
  "quant":   {"weight": null, "activation": "fp32", "output": "fp32"},
  "inputs":  [{"name": "src", "dtype": "fp32", "shape": ["N"], "layout": "contiguous", "access": "read"}],
  "outputs": [{"name": "dst", "dtype": "fp32", "shape": ["N"], "layout": "contiguous", "access": "write"}],
  "dims":    ["N"],
  "params":  [{"name": "_memcpy_bytes", "type": "size_t", "desc": "Number of bytes to copy"}],
  "constraints": {"notes": "Byte copy; performs no arithmetic. src and dst must not overlap."},
  "impl":    {"function": "memcpy", "sources": [], "variants": [{"name": "default", "requires": []}]},
  "call_abi": {"version": 1, "params": [
    {"name": "dst",  "source": "output:dst",     "cast": "void*"},
    {"name": "src",  "source": "activation:src", "cast": "const void*"},
    {"name": "size", "source": "dim:_memcpy_bytes"}
  ]}
}

Now read it top to bottom the way the DSL parser does. Each line is not decoration — it is an instruction to a specific stage of the compiler:

• id — the provider's name in the registry. When this map wins or loses, the X-Ray trace records it by this id. Think of it as the label on the box.
• op — the logical operation it fulfils. The parser only even looks at this map when a circuit asks for residual_save. Wrong op and the map is simply never in the room.
• operation_interface — the hardened contract name the circuit binds to. This is the handshake: the circuit says "I need residual_save.memcpy_copy.v1," and only maps advertising that interface are candidates.
• selection — the eligibility card. status: production means it is allowed to auto-select; phases says it works in prefill and decode; priority breaks ties inside its equivalence group. The parser reads this to decide "may this map play, and how highly is it ranked?"
• quant — the dtypes. The parser filters on this: FP32 activations in, FP32 out. Ask for BF16 and this map is rejected with a recorded dtype_mismatch, not silently used.
• inputs / outputs — the ports, with dtype, symbolic shape, layout, and access. This is what the memory planner and the audit read to prove the data actually lines up and nothing illegally overlaps.
• dims / params — symbolic names (N, _memcpy_bytes) that lowering later resolves to real integers. The map does not hardcode a size; it names one the compiler fills in.
• constraints — the promise in prose: "byte copy, no arithmetic, no overlap." This is where the human-and-audit-readable guarantee lives.
• impl — which C function actually runs (memcpy), and its ISA-gated variants. Here the one default variant requires nothing, so it passes CPU-feature filtering on every machine.
• call_abi — the exact C argument list and where each argument comes from (output:, activation:, dim:). This is the part the code generator turns, mechanically, into a real call.

How The DSL Parser Walks This Map

The v8 DSL (build_ir_v8.py) does not "interpret" the map creatively. It performs a fixed, boring sequence — and boring is the point. For our memcpy map, resolving a residual_save node looks like this:

1. Gather. Collect every map whose op is residual_save.
2. Contract filter. Keep only maps advertising the interface the circuit requires (residual_save.memcpy_copy.v1). Our map survives.
3. Compatibility filter. Check phase (decode? yes), dtype (FP32? yes), layout, and ISA (requires: [], passes everywhere). Any failure here is a recorded rejection reason, never a silent skip.
4. Rank. Among survivors, order by lifecycle status then priority. A tie between two production maps in the same group is a fault, not a coin flip.
5. Bind the ABI. Take the winner's call_abi and resolve each source to a real buffer or dimension, producing the exact C call memcpy(dst, src, size).
6. Emit. Write that call into the generated C. No dispatch remains at runtime — the decision is frozen into the binary.

That is the whole trick. The map carries the knowledge; the parser applies fixed rules to it; and every step that could go wrong produces an inspectable reason instead of a guess. Once you can read one map this way, you can read all 285 — they only differ in how many of these fields they fill in. A pure copy fills in few; a quantized GEMM fills in a numerical contract, ISA variants, and reference oracles too.

The Selection Block: Four Fields That Carry The Weight

Every provider that wants to be auto-selected carries a selection block, enforced by a JSON schema. It has exactly four fields, and each one exists to prevent a specific failure.

The single most important idea here is what priority is not allowed to do. Priority ranks providers only within an equivalence group, and the group name is the arithmetic contract. Look at a real production GEMM map:

"selection": {
  "status": "production",
  "priority": 200,
  "equivalence_group": "q8_0_weight_q8_0_input_llama_fp32_output",
  "phases": ["prefill"]
},
"numerical_contract": "q8_0_weight_q8_0_input_llama_fp32_output",

The equivalence_group and the numerical_contract are the same string: Q8_0 weights times Q8_0 activations producing FP32 output under llama.cpp-compatible semantics. Because priority only ranks within that group, no priority value anywhere can promote a provider with different arithmetic over this one. A tie across groups is not a coin flip — it is a hard fault: "priority cannot choose between different equivalence groups." Priority is allowed to pick the faster of two kernels that compute the same numbers. It is never allowed to pick between two kernels that compute different numbers.

The Resolution Algorithm

With that in place, the actual selection (in _provider_selection_metadata and _rank_provider_matches) is mechanical and boring — which is exactly the point:

1. Validate selection metadata. Every provider must declare an equivalence_group. A missing or malformed selection block is a HARD KERNEL SELECTION FAULT. Providers with no selection block at all are treated as legacy and can never outrank a production provider.
2. Filter by compatibility. Contract identity, phase, dtype, shape, layout, ISA features, and alias safety. Any mismatch rejects the provider — and records why.
3. Rank the survivors. The ranking tuple is direction (inference vs backward) → activation preference → lifecycle rank (production first) → priority (higher wins, within the group).
4. Fail closed on ambiguity. A tie among explicit production providers is a fault. Zero compatible providers is also a compile-time failure. The build never silently substitutes different numerics.

Notice the order: compatibility filtering happens before priority ranking. That ordering is the whole game, and it produces a result that looks backwards until you understand it.

Evidence: The Highest Priority Loses

This is not a story I am telling about the code. It is a fixture the compiler emits. The X-Ray trace in version/v8/tests/fixtures/xray/provider_selection_trace.json records a real selection, including every rejection and its reason.

reason = status_not_production

A priority-900 provider entered the room and lost to a priority-100 provider. Not because 100 beats 900, but because the 900 was a candidate, the 800 was for the wrong phase, and the 700 had the wrong weight dtype — and all three of those checks run before priority is consulted. Higher priority loses to incompatibility every single time. That is the invariant, and it is recorded as evidence, not asserted in a comment.

What Legitimately Stays In The DSL

I want to be honest about what did not move into maps, because the interesting part of any architecture is its boundary. Not everything belongs in a kernel map. Consider residual_save, the byte copy that stashes a residual before a branch.

• Deciding that a residual must be saved is scheduling — it depends on graph shape (a pre-norm followed by a branch) and stays in the DSL, where should_insert_residual_save lives. That is graph knowledge, not kernel knowledge.
• Deciding which copy kernel runs is selection — the memcpy map is selection-managed with declared copy semantics, and wins or loses through the same mechanical resolver as everything else. It is a byte copy, honestly labeled as one, not residual arithmetic hiding under a copy's name.

The debt that remains is named, not hidden. The resolver still carries template-override branches for rope_qk, rope_q, mrope_qk, position embeddings, and a decode KV-store fabricator. Those are migration debt being actively burned down — and, crucially, the burn-down is measured.

The Migration Scoreboard

This is the part I am most attached to, because it is what keeps the whole design honest. kernel_interface_migration_baseline.json is a monotonic ratchet checked in CI: floors may only rise, ceilings may only fall. If a change regresses the migration, the audit fails the build.

The direction is monotonic: physics moves into the maps, heuristics leave the DSL. PR #302 introduced hardened selection. PR #305 migrated shared RoPE, residual-copy, and KV-cache providers. PRs #318 and #320 moved physical layout selection into the maps. The end state is an agnostic DSL — circuits state contracts, maps own everything physical, and the resolver contains no model knowledge at all.

How Kernel Maps Compare To Other Frameworks

Every inference stack has to solve the same underlying problem: given an operation, pick a concrete implementation. What differs is when the decision happens, where the knowledge lives, and what happens when the decision is ambiguous. I want to be fair here — these are all good systems solving slightly different problems — but the contrast is what makes CKE's choice legible.

The differences that matter to me are three. First, CKE resolves at compile time and bakes the choice into generated C, so there is no per-call dispatch cost and no runtime surprise. Second, the selection key is a numerical contract, not just a dtype and device tuple — two providers only compete if they are proven to compute the same arithmetic, which is a stronger identity than "same signature." Third, and most important, CKE fails closed. Most dispatchers treat an unresolved op as a reason to fall back to something; CKE treats it as a reason to stop the build. A fallback that quietly changes your numerics is exactly the class of bug this system exists to make impossible.

None of this makes CKE "better" than PyTorch — PyTorch's dynamic dispatch is the right call for an eager research framework serving thousands of ops on every kind of hardware. It makes CKE different on purpose: it trades flexibility for a compile-time guarantee that the kernel you shipped is the kernel you meant, on the CPU you meant, computing the math you meant.

Is This Over-Engineered?

I ask myself this directly, so I will ask it here. 285 JSON maps, a schema for four fields, a ratchet file, and an audit script can absolutely look like architecture for its own sake. The honest answer has three parts.

Why the layering exists: when provider choice is a Python conditional, the failure mode is silently wrong numerics that pass leaf tests and corrupt a stitched model. Fail-closed selection converts that into a compile-time error with a recorded reason. The costs are real: 285 maps to maintain, 73 legacy conditionals still in the resolver, named debt (RoPE, position embeddings, decode KV store) not yet migrated. Nobody should pretend this is finished or free.

Why This Matters Beyond CKE

The general lesson is not specific to AI compilers. Any system that dispatches among many implementations of the same logical operation faces this choice: put the decision in code, or put it in data with a mechanical resolver. Code is easy to write and impossible to audit at scale. Data with a fail-closed resolver is more work up front and gives you something priceless — every decision and every rejection becomes inspectable evidence, and the dangerous answer ("silently ran the wrong thing") becomes an impossible one ("refused to build, and told you why").

For CKE specifically, this is what lets a new model family land in days instead of weeks. When a bring-up fails, the resolver does not shrug — it names the missing contract or the rejected provider. The compiler tells you what it could not prove, instead of guessing and moving on.