← All posts

Tagged With

machine-learning

9 posts connected to this tag.

Jul 6, 2026

How the Jacobian Matrix Works

.jacobian-matrix-post .math-block { display: block; max-width: 100%; overflow-x: auto; overflow-y: hidden; -webkit-overflow-scrolling: touch; margin: 1rem 0; padding: 0.85rem 1rem; border: 1...

Read post →
Attention: The Core Of The Transformer

Jun 10, 2026

Attention: The Core Of The Transformer

Attention is the core transformer mechanism: Q/K/V projections, head splitting, RoPE, scaled dot-products, masking, softmax, weighted value sums, GQA, Flash Attention, and the full backward ...

Read post →

Jun 5, 2026

Softmax: The Probability Engine

Lab note Companion post to the Softmax carousel. Previously: Matrix Wx+b: From Scalars To Transformers. The previous post was about the moment a scalar weighted sum grew into a matrix multip...

Read post →

Subscribe

Get my rants delivered to your inbox

I will send new posts as and when I write. No fixed cadence, just engineering notes, rants, and things I am thinking through.

Need an intelligent system to work on real hardware?

Embedded systems · Robotics · Constrained AI · CPU and HPC · Accelerators · Distributed systems

Work With Us / Antshiv Robotics

ShivasNotes

Independent notes on software, systems, AI workflows, robotics, and building Antsand in public.

Read

Support