This continues my series on wx+b and backpropagation and matrix multiplication, or GEMM. I want to look at another piece of older mathematics that remains critical in AI: convolution. It was useful in signal processing and image analysis long before today's generative models. Now local filters still help audio systems process sound, vision systems process images, and speech generators build a waveform. The exact role varies by model, but the kernel math is worth understanding.

That range of applications raises the question I want to answer here: what is a convolution actually computing? We will start with a few numbers, move a small weighted sum across a signal, then extend the same idea to an image. From there I will connect the math to GEMM, the C kernels in CKE, and the places convolution still occupies inside modern models. Some architectures use a different projection to meet a similar local-feature need; looking at those variations will also show where compute, memory traffic and training cost enter the design.

Why ask a local question?

Look at a camera image. One pixel tells me a brightness or colour value. A small neighbourhood can tell me whether those values rise, fall, form a corner, or continue a texture. Listen to a recording. One audio sample tells me almost nothing about a phoneme; nearby samples or nearby log-mel frames tell me how the signal is changing. In robotics, one motor-current sample may be harmless, while a short pattern of current and vibration can be worth investigating. Local context changes the question we can ask.

We could give every output location its own completely separate weights, but then the same edge in the top-left of an image and the bottom-right would require different parameters to learn the same local question. Convolution says: use one small set of weights again and again, wherever the window moves. That saves parameters and makes the response map tied to positions in the input. It does not solve recognition by itself. It makes useful local evidence available to later layers.

The older signal-processing version often starts with a hand-designed filter: average neighbouring values, measure a change, emphasize a frequency band. The ML version lets an optimizer learn the filter values from data. The sliding weighted sum is shared. The difference is who chose the weights, what inputs they operate on, and what later stages do with the responses. The SciPy convolution reference is a good fixed-filter comparison, but remember our neural-network forward convention below does not reverse the stored kernel.

One window is still a weighted sum

Let the input be x = [1, 2, 3, 4, 5], the three learned weights be k = [2, 0, -1], and the bias be b = 1. Start at the first three values:

window = [1, 2, 3]
output = 1*2 + 2*0 + 3*(-1) + 1 = 0

Move the same weights one step to the right. The next window is [2, 3, 4], so its output is 1. The last window [3, 4, 5] produces 2. The whole output is [0, 1, 2].

Many deep-learning libraries call this convolution, but their forward operator is technically cross-correlation: the stored kernel is not reversed. If you are checking a formula against a mathematical convolution textbook, this convention matters. The derivative story in part two follows the forward convention written here.

Three ways to use the same 1D operation

Before adding channels and dimensions, I want to see what a filter is actually asking of a signal. A signal can be audio samples, values from a sensor, or one feature measured at successive times. A filter does not know what those numbers mean. It produces a response at every place its small window fits. We give that response an interpretation based on the input, the weights and the rest of the system.

1. Smooth a spike

Suppose a sensor gives [0,0,3,0,0]. With the fixed three-tap average filter [1/3,1/3,1/3], the valid windows give [1,1,1]. The spike no longer appears as one value of three; its contribution is shared across the three output positions whose windows included it. This is the simplest form of a finite-impulse-response (FIR) filter. In real signal processing I would choose the boundary rule and coefficients for the noise and frequency response I need, not assume averaging is always desirable.

2. Find a sudden change

Now take [0,0,0,4,4,4] and the two-tap difference filter [-1,1]. Every constant pair produces zero. The pair [0,4] produces 0*(-1)+4*1=4. The response is [0,0,4,0,0]. Its nonzero position tells me where the rising step is; the sign and size tell me the direction and strength of that change. Swap the filter's sign and the response sign swaps too.

3. Score a local pattern

For [0,1,0,1,0,1,0], use the hand-chosen filter [1,-1,1]. A window [1,0,1] scores 2, while [0,1,0] scores -1. The output [-1,2,-1,2,-1] marks where this particular pattern aligns. This is a response score, not a probability or a recognized word. Its scale depends on the input and filter; a large dot product can also arise from a large-amplitude input. In a learned convolutional layer, training changes the filter values to make responses useful for the task, rather than my choosing these weights by hand.

The graph is not an artist's impression of the answer. The source plots the output of the dot products above. This compact version makes the comparison directly:

import numpy as np
import matplotlib.pyplot as plt

cases = [
    ("average", [0, 0, 3, 0, 0], [1/3, 1/3, 1/3]),
    ("difference", [0, 0, 0, 4, 4, 4], [-1, 1]),
    ("motif", [0, 1, 0, 1, 0, 1, 0], [1, -1, 1]),
]
fig, axes = plt.subplots(3, 2, figsize=(10, 8))
for row, (name, values, weights) in enumerate(cases):
    x, k = np.asarray(values), np.asarray(weights)
    windows = np.lib.stride_tricks.sliding_window_view(x, len(k))
    y = windows @ k
    axes[row, 0].stem(np.arange(len(x)), x)
    axes[row, 1].bar(np.arange(len(y)), y)
    axes[row, 0].set_title(name + " input")
    axes[row, 1].set_title(name + " response")
fig.tight_layout()
fig.savefig("three-1d-uses.png", dpi=180)

import numpy as np

def response(x, kernel):
    windows = np.lib.stride_tricks.sliding_window_view(x, len(kernel))
    return windows @ kernel  # one dot product per window

print(response(np.array([0, 0, 3, 0, 0]), np.ones(3) / 3))
# [1. 1. 1.]
print(response(np.array([0, 0, 0, 4, 4, 4]), np.array([-1, 1])))
# [0 0 4 0 0]
print(response(np.array([0, 1, 0, 1, 0, 1, 0]), np.array([1, -1, 1])))
# [-1  2 -1  2 -1]

This fixed-filter view is the signal-processing ancestor of a learned convolution. A microphone recording is a time series; so are motor current and inertial measurements on a robot. Smoothing, change detection and local pattern matching are useful long before a neural network enters the picture. A learned model adds many filters, channels, nonlinearities and a training objective, but it still starts with the same local multiply-and-add. Same sliding sum. The filter can be designed by a person or learned from examples.What changes between signal processing and ML

Why 1D? Why 2D?

A 1D convolution moves along one axis. For audio features, that axis is usually time. A 2D convolution moves a small rectangle over height and width, as in an image or spectrogram. The learned operation inside a window is the same dot product. A 2D filter of height 3 and width 3 is simply nine weights per input channel before we account for output channels.

The axes have to be explicit. A common 1D activation shape is [batch, input_channels, time]; the filter shape is [output_channels, input_channels, kernel_width]. A common 2D activation shape is [batch, input_channels, height, width]; the filter shape is [output_channels, input_channels, kernel_height, kernel_width]. Each output channel has its own filter spanning all input channels. A depthwise convolution is a different, cheaper contract: it applies a separate spatial filter to each channel rather than mixing all channels in that step.

Here is a tiny 2D version with one input channel and one output channel. Put a 2x2 filter on the upper-left 2x2 image window, multiply matching entries, and sum. With stride one and no padding, the 3x3 image gives a 2x2 result:

image  = [[1, 2, 0],    filter = [[1,  0],
          [3, 4, 1],              [0, -1]]
          [0, 2, 5]]

top-left output = 1*1 + 2*0 + 3*0 + 4*(-1) = -3
all outputs     = [[-3,  1],
                   [ 1, -1]]

What does that -3 mean? Only that the upper-left patch gave a negative response to this filter. It does not mean “not an image” or “not an edge.” The filter compares its top-left value with its bottom-right value; here the latter is four units larger. The response at each output cell is attached to the patch that produced it. Sliding the filter creates a feature map: a spatial map of where this particular question got which answer.

Build an image-gradient map, one number at a time

Image processing gives us a more visual example. Imagine a six-by-six grayscale image: zeros on the left, ones on the right. The vertical line where dark becomes bright is its only edge. A horizontal-change filter, often called Gx, compares left and right values in each 3x3 patch. A vertical-change filter Gy compares top and bottom. These are fixed, Sobel-like example filters, not learned CKE model weights.

image X                    Gx filter        Gy filter
0 0 0 1 1 1               -1  0  1        -1 -2 -1
0 0 0 1 1 1               -2  0  2         0  0  0
0 0 0 1 1 1               -1  0  1         1  2  1
0 0 0 1 1 1
0 0 0 1 1 1
0 0 0 1 1 1

Place both filters on the top row, starting at the second column. The patch they read is three copies of [0,0,1]. For Gx, the top row contributes 1, the middle contributes 2, and the bottom contributes 1: total 4. For Gy, the top contributes -1, the middle contributes 0, and the bottom contributes +1: total 0. We found a change across the horizontal coordinate, so Gx responds to a vertical-looking edge; Gy does not.

Gx output [4x4]             Gy output [4x4]
0 4 4 0                    0 0 0 0
0 4 4 0                    0 0 0 0
0 4 4 0                    0 0 0 0
0 4 4 0                    0 0 0 0

edge strength = sqrt(Gx*Gx + Gy*Gy)
              = the same [0,4,4,0] row, repeated four times

import numpy as np
import matplotlib.pyplot as plt

image = np.zeros((6, 6))
image[:, 3:] = 1
gx_filter = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]])
gy_filter = np.array([[-1, -2, -1], [0, 0, 0], [1, 2, 1]])
patches = np.lib.stride_tricks.sliding_window_view(image, (3, 3))
gx = np.einsum("ijab,ab->ij", patches, gx_filter)
gy = np.einsum("ijab,ab->ij", patches, gy_filter)
strength = np.hypot(gx, gy)

fig, axes = plt.subplots(2, 2)
for ax, values, label in zip(axes.flat,
        (image, gx, gy, strength), ("input", "Gx", "Gy", "strength")):
    ax.imshow(values, cmap="gray" if label == "input" else "YlOrBr",
              vmin=0, vmax=1 if label == "input" else 4)
    ax.set_title(label)
    for row in range(values.shape[0]):
        for col in range(values.shape[1]):
            ax.text(col, row, f"{values[row, col]:.0f}",
                    ha="center", va="center")
fig.savefig("sobel-numbered-matrices.png", dpi=180,
            bbox_inches="tight")

The result has shape 4x4 because we used a valid 3x3 filter: there are four possible starting rows and four possible starting columns in a 6x6 input. We did not pad the outside. If we padded, the size and edge values would change. Those choices matter when we compare a C kernel with an oracle or try to stitch layers together.

“Gradient” here means a spatial image gradient: how pixel intensity changes across x or y. It is not yet the loss gradient used to train a model. The next article in this series will deal with that different derivative. Two gradientsSobel measures change across pixels. Backprop measures how a loss changes with a weight or activation.

For multiple channels, each window includes a small patch from every input channel, and their products are summed into one output channel. If there are Cin input channels and a Kh x Kw filter, one output filter has Cin*Kh*Kw weights plus one bias. With Cout output filters, the complete parameter count is Cout*(Cin*Kh*Kw + 1), independent of the image's height and width. That independence is what spatial weight sharing buys you.

With no padding, stride one and dilation one, a length-five input and width-three filter give 5 - 3 + 1 = 3 positions. More generally, for one axis:

effective_kernel = dilation * (kernel_width - 1) + 1
output_length = floor((input_length + 2*padding - effective_kernel) / stride) + 1

Padding inserts values at the edge, usually zeros. Stride chooses how far to move between windows. Dilation spaces the taps of the filter apart. Apply the same output-size rule independently to the height and width axes in 2D. The formula is a shape contract, not merely notation: an off-by-one error here changes every later tensor.

See the signal, filter and image

The first plot below shows exactly which samples each output reads. The second uses a tiny grayscale image and a simple horizontal-difference filter. The example filter is hand-chosen to make an edge visible; it is not a trained model weight.

import numpy as np
import matplotlib.pyplot as plt

x = np.array([1., 2., 3., 4., 5.])
k = np.array([2., 0., -1.])
windows = np.lib.stride_tricks.sliding_window_view(x, 3)
y = windows @ k + 1

fig, (signal_ax, output_ax) = plt.subplots(2, 1)
signal_ax.plot(range(len(x)), x, "o-")
output_ax.bar(range(len(y)), y)
signal_ax.set_title("Input and overlapping windows")
output_ax.set_title("One output per window")
fig.savefig("sliding-window.png", dpi=180, bbox_inches="tight")

image = np.zeros((12, 12))
image[3:9, 4:9] = 1
edge_filter = np.array([[-1, 0, 1], [-1, 0, 1], [-1, 0, 1]])
patches = np.lib.stride_tricks.sliding_window_view(image, (3, 3))
response = np.einsum("ijab,ab->ij", patches, edge_filter)

fig, axes = plt.subplots(1, 2)
axes[0].imshow(image, cmap="gray")
axes[1].imshow(response, cmap="RdBu_r", vmin=-3, vmax=3)
fig.savefig("image-filter.png", dpi=180, bbox_inches="tight")

The plotting snippets above compute their displayed responses rather than drawing expected numbers by hand. The longer source used for these figures also checks the tiny 2D calculation numerically.

The matrix connection

Remember the earlier matrix Wx+b post: stack inputs into rows and evaluate many weighted sums as a matrix multiply. We can do the same to convolution. Write each local window as one row:

X_windows = [[1, 2, 3],
             [2, 3, 4],
             [3, 4, 5]]       # [positions, kernel_width]
k         = [2, 0, -1]      # [kernel_width]
y         = X_windows @ k + 1
          = [0, 1, 2]

What does the @ do? It takes one row and one column, multiplies corresponding values and sums them. With the first window and our filter, [1,2,3] @ [2,0,-1] = 2 + 0 - 3 = -1; adding bias one gives output zero. Repeat that dot product for the other two rows. This is not a different answer from sliding the filter. It is the same answer arranged so a matrix engine can do many outputs together. GEMM means general matrix multiplication; it is the well-studied operation behind that @.

Suppose we want two features at each 1D position. Put our difference-like filter in column zero and a sum filter [1,1,1] in column one. The two columns ask two different questions of exactly the same three windows:

positions x taps      taps x filters     bias         positions x filters
X = [[1,2,3],        K = [[ 2,1],     b = [1,0]    Y = [[0, 6],
     [2,3,4],             [ 0,1],                      [1, 9],
     [3,4,5]]             [-1,1]]                      [2,12]]

shape: [3,3] @ [3,2] + [2] = [3,2]
row 0, filter 0: 1*2 + 2*0 + 3*(-1) + 1 = 0
row 0, filter 1: 1*1 + 2*1 + 3*1 + 0 = 6

The rows of Y still identify positions. Its columns identify filters. Column zero is the response map [0,1,2]; column one is [6,9,12]. Nothing in those numbers says “cat,” “speech,” or “safe flight.” The network can later combine many feature maps, apply an activation, reduce resolution or use attention. The original spatial or temporal meaning comes from how we map those rows back to their positions.

Now return to our 6x6 image-gradient example. Each valid 3x3 patch contains nine numbers. There are 4*4=16 patch positions. Flatten each patch into a row of a [16,9] matrix. Flatten Gx and Gy into two columns of a [9,2] filter matrix. Then:

patch matrix       filter matrix      response matrix
P [16 x 9]    @    K [9 x 2]     =     R [16 x 2]

patch at output (row=0, col=1):
P row = [0,0,1, 0,0,1, 0,0,1]
Gx col = [-1,0,1, -2,0,2, -1,0,1]
Gy col = [-1,-2,-1, 0,0,0, 1,2,1]
P row @ Gx col = 1 + 2 + 1 = 4
P row @ Gy col = -1 + 0 + 1 = 0

reshape R [16 x 2] -> [4 x 4 x 2]  # two spatial feature maps

This window-unfolding technique is often called im2col. It can reuse fast GEMM kernels, but storing every overlapping patch duplicates input values. Our 6x6 image has 36 values; the materialized patch matrix has 16*9=144 values. That is four times as many elements in this small example, before filter or output storage. A direct or fused convolution kernel can compute from the original image without materializing the whole patch matrix. The equation establishes equivalence; the implementation strategy is a separate performance decision. 36 to 144The matrix view proves the arithmetic, not that a kernel must allocate four times the input elements.

A 1x1 convolution is the limiting case: its window has one spatial location. It mixes channels at each position but does not, by itself, mix neighbouring positions. That is a useful bridge to the linear projections already familiar from Wx+b.

What a response represents after the patch moves

Here is the interpretation I would keep beside the math. The input patch is a location and its nearby measurements. The filter is a question about those measurements. The output cell is that question's numerical response at that location. Sliding creates a response map. Multiple filters create multiple maps, which become the next layer's input channels. A negative response is not automatically an error; an activation such as ReLU may choose to suppress it, but that happens in a separate operation. Read the activation-function note after this one if you want to see why that extra step matters. A strong response means this filter matched here. It is not yet an object label, word, or decision.How to read a feature map

For an image, location means height and width. Our fixed Gx filter produces a high score around a vertical intensity boundary. A learned first-layer filter might instead respond to a colour combination, oriented texture, or something less easy to name. Deeper layers can combine those maps into more useful representations. Neither our edge map nor a learned first-layer map is, by itself, a claim that the image contains an object. The CKE vision boundary note is a more applied example of why the exact encoder output and the next connection must be checked, not inferred from a plausible-looking image response.

For audio, location might mean time in a 1D sequence, or time and frequency in a 2D log-mel spectrogram. A time filter can respond to a local acoustic pattern and, with stride, reduce the number of time positions. A 2D filter on a spectrogram can read a small neighbourhood of both time frames and frequency bins. A high score is still only a feature response, not a transcribed word or a speaker identity. Whisper's transformer and decoder do much more work after its convolutional stem; the Whisper kernel walkthrough follows that pipeline from audio features toward words.

That gives one useful way to remember 1D versus 2D: the number refers to axes the filter moves over, not how many dimensions the full tensor has. A 1D audio activation still has batch and channel dimensions. A 2D image activation still has batch and colour/feature channels. And a spectrogram is two-dimensional even though it came from a one-dimensional recording.

Where this shows up in models I work with

Convolution did not end when transformers arrived. Its job depends on where it sits in the model. Sometimes it prepares local measurements for attention. Sometimes it mixes neighbouring frames inside an encoder. Sometimes it helps turn a generated representation back into sound. The same sliding dot product can serve each job, but the tensor layout, padding, stride and surrounding operations are different.

Speech going in: Whisper and FastConformer

Whisper's audio encoder applies two learned Conv1D layers to log-mel features before its transformer body. The first mixes nearby time frames; the second uses stride two, so the transformer receives a shorter sequence. Attention is still doing the longer-range work, but convolution prepared the local features and changed the time resolution. The CKE audio kernel deep dive follows this precise layout boundary. In FastConformer, convolution also appears in the subsampling path and inside encoder blocks. That is not a nostalgic CNN bolted on to a transformer; local filtering and global mixing have different jobs.

Speech coming out: Kokoro and Qwen3-TTS

Generation reverses the direction of the question. Kokoro begins with phonemes and produces acoustic output. Its decoder and iSTFT generator source contains Conv1D residual blocks and transposed Conv1D upsampling. The ordinary Conv1D pieces mix neighbouring time positions; transposed convolution increases time resolution. They are related operators, but transposed convolution is not simply the same forward filter run backwards. Style, pitch and other branches join that path before it produces spectral information and an inverse STFT turns that into a waveform. The Kokoro anatomy note and ALBERT phoneme-encoder note separate the text side from this output side.

Qwen3-TTS's official speech-tokenizer source gives another modern example: it defines causal Conv1D, depthwise causal convolution in a ConvNeXt-style block, and transposed Conv1D for upsampling in its tokenizer decoder. Causality matters because a generated time step should not read future audio. This is evidence about Qwen3-TTS's published model code, not a claim that CKE currently runs that entire TTS model. It shows why learning one small 1D filter is still useful when reading a much larger generative system.

Images going in: two ways to make patches

Conv2D remains important in vision, but I would not say every vision encoder uses it. A CNN-style encoder can stack learned spatial filters. A transformer can also use Conv2D as a patch-projecting stem: the Torchvision Vision Transformer implementation uses a Conv2D projection with kernel size and stride equal to patch size. That is a particularly clear case: each non-overlapping image patch becomes an embedding before transformer blocks process the resulting token sequence. By contrast, SigLIP 2's vision embedding implementation applies a linear layer to already-patchified pixels. The original Vision Transformer paper describes flattened patches and linear projection. A Conv2D implementation and a patch-matrix projection can express equivalent arithmetic in a restricted non-overlapping case, but their actual runtime and preprocessing paths must be inspected, not assumed.

Why variants matter: work, bytes and capacity

The common need is to turn a local piece of signal into a useful feature. There is more than one way to do that, and the useful question is not merely “which formula is newer?” It is: how many multiply-adds and parameters does this choice need, how many bytes does it move, what does training need to retain for backward, and does the resulting representation still do the job? These are different measures. A kernel can do fewer operations yet run slower if its memory access is poor; it can run fast in inference but require more saved activations during training.

Take a non-overlapping P x P image patch with Cin input channels and D output features. A Conv2D with kernel P x P and stride P computes D * Cin * P * P weight products per patch. Flattening the same patch and applying a [Cin*P*P, D] linear weight matrix computes the same count. With corresponding weights and bias, the two projections can produce the same values, apart from floating-point reduction-order differences. SigLIP 2's linear patch projection is therefore not automatically a mathematical or FLOP-saving breakthrough over that particular Conv2D stem. Its input is already patchified; the cost of producing and moving those patches belongs in the end-to-end comparison. A fused convolution might avoid storing a patch matrix, while a linear path may map well to a tuned GEMM. Which wins depends on shapes, layout and hardware.

Other variations really do trade expressive freedom for less work. A dense 3x3 Conv2D from 64 channels to 64 channels has 64*64*3*3 = 36,864 weights before bias. A depthwise 3x3 pass followed by a pointwise 1x1 channel-mixing pass has 64*3*3 + 64*64 = 4,672 weights: about 7.9 times fewer, and roughly the same reduction in multiply-adds per output position if we ignore bias and edge handling. But depthwise-plus-pointwise is a constrained factorization, not an exact replacement for every possible dense filter. Similarly, a stride-two audio stem reduces the sequence length passed to later layers, but it also discards resolution. An architect chooses these variants because the quality, compute and memory trade-off is acceptable for a specific task, not because all local-feature operators are interchangeable.

The im2col example above makes the memory side concrete: materializing 16 overlapping 3x3 patches from our 6x6 image creates 144 patch values from 36 input values. A direct kernel can avoid that intermediate; a GEMM-based route may recover time through better matrix hardware use. During training, saving intermediates or recomputing them adds another choice. This is the CPU-kernel question I want CKE to make inspectable: keep the requested mathematical contract, then measure arithmetic, memory traffic, intermediate storage and numerical error for each implementation. Same purpose does not always mean identical math. Exact patch projection is one equivalence; depthwise factorization is a deliberate restriction.Check the operator contract before comparing speed

Where the kernel joins the larger model

Here is the connection I wish I had seen when first learning convolution. Our three-number 1D filter and nine-number 2D filter are not complete AI systems. They are local operations inside a chain. Each stage has a contract: input shape and layout, filter weights, output shape, boundary rule, and the next operation that consumes its response. The model becomes useful by stitching many such operations together, then training or otherwise selecting their weights.

Whisper ASR:  waveform -> log-mel -> Conv1D stem -> attention -> text decoder
Kokoro TTS:   phonemes -> text/prosody features -> Conv1D decoder and upsampling -> iSTFT -> waveform
Qwen3-TTS:   generated audio codes -> tokenizer decoder's causal Conv1D/upsampling -> waveform
Vision:      pixels/patches -> Conv2D stem OR linear patch projection -> visual tokens -> attention

The arrows hide necessary work. None of these paths is only a convolution.

That is how I want to think about kernels in CKE: first understand the mathematical operator and tensor contract, then inspect where a model uses it, then select an implementation, stitch its output to the next circuit operation and check the numbers at that boundary. CKE's audio Conv1D and Conv2D providers are concrete examples below. They do not automatically become a Kokoro, Qwen3-TTS or arbitrary vision runtime merely because those models contain a convolution. The exact layouts, weights, schedules and other kernels must connect too. This is why the algorithm is still current in generative AI, and why a reusable, testable kernel matters more than a demo that happens to produce plausible output.

Find the same sum in CKE's C kernels

There is an especially useful sanity check in the CKE audio kernel source. In audio_conv1d_channel_major_grouped_f32, the worker visits an output channel and frame, starts an accumulator at the bias, then loops over the input channels and filter taps belonging to that group. Each valid tap reads one input frame and one weight and accumulates their product. The essential scalar path reduces to this:

float sum = bias ? bias[out_channel] : 0.0f;
for (int local_channel = 0; local_channel < channels_per_group;
     ++local_channel) {
    int in_channel = group_begin + local_channel;
    for (int tap = 0; tap < kernel_width; ++tap) {
        int frame = out_frame * stride + tap - padding;
        if (frame >= 0 && frame < input_frames) {
            sum = fmaf(input[in_channel][frame],
                       weight[out_channel][local_channel][tap], sum);
        }
    }
}
output[out_channel][out_frame] = sum;

That is pedagogical pseudocode, not a copy-paste replacement for the real kernel's flat-buffer indexing, grouping or SIMD path. The real source also has an AVX2/FMA interior path and explicit boundary handling. Read the 1D implementation and then compare the loops to our sum(input * weight) + bias. The compiler and circuit still have to supply the right shape, weight layout, stride, padding and buffer; a mathematically correct isolated kernel is not proof that an entire model is correct.

The same file also contains audio_conv2d_whc_grouped_f32. It adds nested loops over kernel height and width and maps output coordinates back to input coordinates. CKE calls this 2D kernel for a FastConformer audio subsampling path over time and feature dimensions. It is a concrete C example of the same two-axis arithmetic we drew for an image, not evidence that this audio-specific function is automatically the provider for every vision encoder. The CKE FastConformer kernel diagram shows how pointwise, depthwise, normalization and activation stages fit together after subsampling.

There is a different 1D example in ssm_kernels.c: the ssm_conv1d_forward_ref path applies a short causal, depthwise filter to each channel before a recurrent state update. Here “causal” means a token's response cannot read future tokens. The underlying multiply-and-sum is familiar, but the boundary condition and saved history are part of the model contract. This is one reason convolution has not been replaced wholesale by attention: it can serve local time structure inside quite different architectures.

CKE's direct C loops need not build our [16,9] patch matrix. The matrix view explains which multiplications must happen; the C code chooses a memory and parallelism strategy. If you want to go another layer down into exact function contracts, the CKE kernel registry note is the next link.

Why convolution is still important

Attention can connect distant positions, but it does not make local structure disappear. A short convolution brings neighbouring samples together before a model spends work on a longer-range representation. In Whisper, those learned audio filters both transform the features and, through the second layer's stride, shorten the sequence that attention receives. In image systems, a local filter can extract an edge or texture; a patch projection can turn a region into one token. In speech generation, convolutional blocks can operate on the time structure of the signal that will become a waveform. These are different jobs, not one universal recipe.

There is also an engineering reason to keep studying the operator. A small shared filter has a parameter count that does not grow with image size or recording length. The output work still grows with the number of positions, so it is not free. But the repeated, local arithmetic gives a kernel engineer concrete choices: direct computation, blocked layouts, channel grouping, depthwise filtering, im2col plus GEMM, or a fused path. On a CPU I can inspect whether the implementation is reading data efficiently, reusing cache lines, and preserving the numerical result. That is the same practical question I ask of CKE's other kernels: what computation does the model require, and what is the least wasteful way to execute it on the hardware I own?

Weight sharing also gives a useful structural property. With stride one and away from boundary effects, if the same local pattern moves one position in the input, its response moves one position in the output. That is translation equivariance, not automatic translation invariance: the location still matters, and padding, stride and later layers can change the relationship. For images, audio and sensor streams, being able to reuse a local question at many positions is still valuable. Attention answers a different question about relationships between positions. Modern models often compose both; the actual architecture tells us when they do.

Related Notes and Sources

• Backpropagation From wx+b: The Smallest Training Loop
• Matrix Wx+b: From Scalars To Transformers
• Activation Functions: The Nonlinearity Inside Neural Networks
• How Audio Becomes Words: Whisper's Kernels
• How Kokoro TTS Works and Where CKE Stands
• CKE Audio Kernels Deep Dive
• Whisper reference model implementation
• Kokoro decoder and iSTFT generator implementation
• Qwen3-TTS speech-tokenizer convolution implementation
• Torchvision ViT Conv2D patch-projection implementation
• SigLIP 2 linear patch-embedding implementation
• PyTorch Conv2d: stride, groups and depthwise contracts
• PyTorch Conv1d: cross-correlation, shapes and parameters
• SciPy signal convolution: full, same and valid output modes
• A Guide to Convolution Arithmetic for Deep Learning