← All posts

Tagged With

LoRA

2 posts connected to this tag.

LoRA & QLoRA: Inside the Update

Nov 13, 2026

LoRA & QLoRA: Inside the Update

When to fine-tune, how PEFT, LoRA and QLoRA differ, and where adapters fit inside attention and the MLP—with worked matrices, gradients, memory charts and a practical training plan.

Read post →

Follow the work. Understand the machine.

I write about the maths, the kernels, and what happens when I try to make them work. Join me for the investigations, including the bits that go wrong.

ENGINEERING NOTES FROM ANTHONY SHIVAKUMAR
How to think like an AI kernel and HPC engineer — ShivasNotes field guide cover
THE ILLUSTRATED FIELD GUIDE

How to think like an AI kernel & HPC engineer

  • 10 pages to connect the pieces
  • A real Whisper investigation
  • A worksheet for your next experiment
Take a look inside ↗
KEEP IN TOUCH

A note when there’s something to share.

Get the illustrated field guide by email, followed by engineering notes as I write them.

  • You’re joining the ShivasNotes email list. No fixed cadence.
  • Your welcome email includes the PDF. Allow a few minutes, and check spam if it hasn’t arrived.

Need an intelligent system to work on real hardware?

Embedded systems · Robotics · Constrained AI · CPU and HPC · Accelerators · Distributed systems

Work With Us / Antshiv Robotics

ShivasNotes

Independent notes on software, systems, AI workflows, robotics, and building in public.

Read

Support

Antshiv Robotics

Owned by Antshiv Robotics

ShivasNotes is an Antshiv Robotics Inc. publication.

Antsand

Published with Antsand

The publishing platform behind ShivasNotes.