System 08

Fine-Tuning Pipeline with LoRA

Pick an adapter size and watch a small model learn a new way of talking, then check what the shortcut cost.

What it does

A support inbox sorts messages into six buckets: billing, password reset, cancellation, order tracking, bug report, feature request. The starting model was trained on polite, well-punctuated writing. The messages it now has to handle are clipped, lowercase and full of typos, so it needs to adapt.

There are three ways to do that, and this page runs all three on the same test messages. Ask the model directly with no training at all. Retrain every weight it has. Or freeze the whole thing and train a tiny add-on beside it, which is what LoRA means. You choose how big that add-on is by typing a number, and the page shows how many settings each approach had to change and how well each one scored.

The last check is the one people skip. The add-on can be folded back into the original weights with plain arithmetic, so the finished model is one plain matrix again with nothing bolted on and nothing slower about it. The page proves the folded model gives exactly the same answers, to nine decimal places.

Everything trains here and now in a couple of seconds. It is small on purpose, and the output says exactly what was shrunk to make that possible.

Try it

41816