New Action 3

Jul 18, 2026

CKE Kernel Registry: How Exact C Functions And Numerical Contracts Connect

How CKE resolves model circuits to exact C kernel providers, preserves numerical contracts through IR, and generates correct stitched code.

Read post →

Jul 7, 2026

DeepSeek-V4 Architecture From Kernels: mHC, CSA, HCA, DSA, and C-Kernel-Engine

A kernel-level reading of DeepSeek-V4: mHC residual lanes, CSA/HCA compressed attention, DSA sparse selection, MoE dispatch, low precision, and what those ideas mean for C-Kernel-Engine.

Read post →

Jul 4, 2026

DeltaNet vs SSM: Two Fixed-Size Memory Systems Beyond Attention

C-Kernel-Engine architecture note DeltaNet and SSM/Mamba are two ways to reduce the amount of unbounded attention history a model has to carry by introducing fixed-size learned state. They a...

Read post →

Jun 25, 2026

KV Cache Memory: The Hidden State That Makes LLM Decode Work

KV cache memory for CPU-native inference This ShivasNotes deep dive is written for engineers who want to understand the single largest memory consumer in autoregressive LLM inference: the KV...

Read post →

Jun 21, 2026

Flash Attention on CPU: Online Softmax, Cache Discipline, and C-Kernel-Engine

ML fundamentals This ShivasNotes deep dive is written for CPU silicon teams asking a very specific question: does C-Kernel-Engine really own attention on CPU down at the kernel layer? The an...

Read post →

Jun 17, 2026

How Qwen3-VL Vision Works: Header Body Footer

Lab note This ShivasNotes lab note studies how a real vision-language model turns images into tokens that a language decoder can use. It connects earlier notes on activations, normalization,...

Read post →

Jun 16, 2026

Gated DeltaNet: Recurrent Attention With Error-Driven Memory

Lab note This ShivasNotes lab note studies the alternative used by the Qwen3.5 / qwen3next recurrent path in C-Kernel-Engine: Gated DeltaNet, where the model stops trying to store every toke...

Read post →

Jun 10, 2026

Attention: The Core Of The Transformer

Attention is the core transformer mechanism: Q/K/V projections, head splitting, RoPE, scaled dot-products, masking, softmax, weighted value sums, GQA, Flash Attention, and the full backward ...

Read post →

Jun 8, 2026

Positional Encoding: Teaching Transformers Where To Look

Lab note Companion post to the Positional Encoding carousel. Previously: LayerNorm And RMSNorm: Stabilizing The Signal. Transformers are brilliant at reading an entire sequence in parallel, ...

Read post →

Jun 5, 2026

Softmax: The Probability Engine

Lab note Companion post to the Softmax carousel. Previously: Matrix Wx+b: From Scalars To Transformers. The previous post was about the moment a scalar weighted sum grew into a matrix multip...

Read post →

attention

CKE Kernel Registry: How Exact C Functions And Numerical Contracts Connect

DeepSeek-V4 Architecture From Kernels: mHC, CSA, HCA, DSA, and C-Kernel-Engine

DeltaNet vs SSM: Two Fixed-Size Memory Systems Beyond Attention

KV Cache Memory: The Hidden State That Makes LLM Decode Work

Flash Attention on CPU: Online Softmax, Cache Discipline, and C-Kernel-Engine

How Qwen3-VL Vision Works: Header Body Footer

Gated DeltaNet: Recurrent Attention With Error-Driven Memory

Attention: The Core Of The Transformer

Positional Encoding: Teaching Transformers Where To Look

Softmax: The Probability Engine

Need an intelligent system to work on real hardware?

Embedded systems · Robotics · Constrained AI · CPU and HPC · Accelerators · Distributed systems

ShivasNotes

Read

Support

Subscribe

Subscribe to emails from Anthony

Need an intelligent system to work on real hardware?

Embedded systems · Robotics · Constrained AI · CPU and HPC · Accelerators · Distributed systems

ShivasNotes

Read

Support