Fonzie: three weeks of teaching a 4B model when to look things up

Qwen3.5-4B → Fonzie0.2-4B, 5–26 October 2026. What I tried, what broke and what finally worked.

I wanted a small model with three goals:

1. Don't invent facts; search when you don't know. This is the first principle. The model has to know where its knowledge ends. If it knows the answer, it answers. If it doesn't, it searches, cites what it found, and says plainly when the search comes back empty.
2. Speak Italian without mistakes, and answer in the language of the question.
3. Recognise what's in a photo, like a historical monument or a painting, better than the base model, and check with a search when unsure.

And it had to do all of this inside the agent apps I use: oh-my-pi, opencode and Hermes.

So I spent three weeks fine-tuning Qwen3.5-4B on a DGX Spark. The result is Fonzie0.2-4B. On a test set I built for the things the base model got wrong, it answers 23–26 points more questions correctly and makes 27–31 points fewer confident mistakes, in all three apps.

Fonzie0.2 runs in a few GB of memory. Besides the full bf16 model, I made two GGUF versions for llama.cpp, LM Studio and the other apps built on llama.cpp: Q4_K_M (2.8 GB) and Q8_0 (4.6 GB), plus a 0.7 GB vision projector for photos. How much memory the server needs depends mostly on the context you configure:

This is the GPU memory llama-server uses with the vision projector loaded, MTP on and the default f16 KV cache. I measured the Q4_K_M on my RTX 4070 Ti at 64k, 128k and 160k. The other Q4_K_M rows add 38 KB per token of context to the measured fixed part (3.9 GiB). The Q8_0 rows add its bigger file (1.7 GiB more) to the Q4_K_M numbers; I haven't run the Q8_0 myself. A q8_0 KV cache takes less than half the memory per token: with it, the Q4_K_M fits 229,376 tokens on the 12 GB card, peaking at 10.0 GiB. Desktop apps hold some memory on the same card, so leave a margin.

The Q4_K_M loses some quality. In opencode and Hermes it scores like the bf16, within the noise. In oh-my-pi it gets about 13 points fewer everyday questions right, and on the frozen test it invents more links (7.2% of answers against 2.6%). The full comparison is near the end, under "How much does the Q4_K_M lose?". I haven't evaluated the Q8_0 yet.

Getting there took about ten versions and a reinforcement learning detour that looked great in my own test and fell apart inside the real apps. Along the way I hit a long list of bugs, most of them mine. The mistakes taught me more than the wins did, so they stay in the post.

One of the main things small models miss is the harness they run in. Each agent app has its own system prompt, its own tools with their own names and schemas, and its own way of returning results. A behaviour Fonzie learned in one format didn't carry over to another. With the same weights, the model from that RL detour searched for 85% of the facts it didn't know in my own test and for 57% inside oh-my-pi. So I recorded what oh-my-pi and opencode really send and generated the training data inside those exact formats, with the same tools and the same prompts. Behaviour inside those apps improved. On the new test set, accuracy rose from 29.2% to 55.4% in oh-my-pi and from 36.1% to 61.1% in opencode, and confident wrong answers fell by about 30 points in both. Hermes, which I kept out of training on purpose, improved too, from 45.0% to 67.8%, so part of the behaviour carries over to an app the model never saw.

The short version

• The base model's problem was calibration. Qwen3.5-4B already calls tools well, sees images and speaks Italian. But it searched for half the questions it could answer from memory, looped on 30% of them, and answered Italian questions in English.
• Data fixed almost everything. Each improvement came from changing the training data: what the teacher saw, which examples the filters kept, which failures I injected. None came from tuning hyperparameters.
• RL won in my test format and lost in the real apps. GRPO gave my best numbers in week one, but inside oh-my-pi the same model searched for 57% of the facts it didn't know instead of 85%. I paused RL and distilled its behaviour into SFT data.
• Train and test where the model will live. My own harness misled me for a week. The model's real job is inside agent apps, each with its own system prompt, tools and quirks, so the training data has to come from inside them too.
• Noise is 3–5 points. Two trainings with identical data, differing only in the seed, ended 3–4 points apart. My evaluation runs vary by about 5 points between repeats. Most "improvements" smaller than that are not real.

The whole process on one page covers how the training data is built, what Fonzie0.1 changed against the base, what 0.2 added to fix 0.1, and the results of all three:

Real demo in opencode

Fonzie0.2-4B (Q4_K_M, MTP 2) in opencode 1.19.4 on an RTX 4070 Ti, with questions in Italian. Something it knows: it answers straight away, without searching. News from 2026: it searches, reads the Wikipedia page and cites it. A false premise: it searches and corrects it. A painter who doesn't exist: it searches, finds nothing and says so. Best of 3 to 6 takes per question, with the model's waits sped up 4×.

Who did what

This was a one-person project with a lot of help, from models and from people.

A few of these need more than a line.

Javier (@javierbits). Most of my method comes from his threads on X:

• A pirate fine-tune of a 4B on a DGX Spark. It showed me the mechanics were within reach.
• Javi0.2-4B. Almost the same project as mine: Qwen3.5-4B, a LoRA, a DGX Spark, and a model that reaches for tools when it doesn't know.
• His rubric → generate → filter → train → eval loop. It became my process.

I adopted his lessons on day one:

• test in the harness you will deploy in;
• check facts with code before asking an LLM judge;
• show the student the same instructions the teacher saw;
• train on the decisions;
• fix eval failures with data, not hyperparameters.

Later, his Javi0.3 thread shaped my v4 almost entirely. If you want to do something like this, read him first.

Unsloth. Every training run in this post is an Unsloth run. I started from their Qwen3.5 fine-tuning guide and their Qwen3.5 vision notebook, then adapted them. Their guide steers you away from 4-bit QLoRA on Qwen3.5, so I trained a bf16 LoRA. It also has sensible defaults for learning rate and epochs, and loss on the answers only. All of that held up.

Claude Code with Claude Opus 5.5 was my lab assistant. It:

• wrote most of the scripts;
• ran the jobs on the Spark over SSH and read the logs;
• kept two documents per lab: a diary, and a decision log in which every important choice gets an entry with what, why, evidence and rejected alternatives.

I made the calls on direction. It did most of the typing and caught a good share of the bugs. It also caused a few. It once restarted services without checking what else was using the memory, and since then it checks before touching anything on the Spark.

On the teacher choice. On day two I planned to use Claude Opus 5.5 as the teacher. Anthropic's terms don't allow training competing models on Claude's output, which would have kept Fonzie off Hugging Face for good. So I went back to local teachers. Qwen3.8 Flash Next and GLM-5.3-Flash both run on the Spark, cost nothing per example, and let me publish the data and the model.

On search. The public search engines blocked the Spark's IP within a day. So the search tool queries a local Wikipedia index: Italian and English, with full-text search and BM25 ranking. It's fast, free and the same every time. It also has no news newer than the dump, which shapes some results below.

Step zero: measure the base model

Before training anything, I wrote a rubric (RUBRIC.md):

• search when unsure, answer directly when sure;
• cite what you used;
• admit when a search fails;
• don't loop;
• answer in the user's language.

The same rubric drove the teacher's prompt, the filters and the eval.

Then I froze a test set of 250 questions that never goes near training:

• facts the 4B knows;
• recent or obscure facts it can't know;
• images to read;
• landmark photos that need a search;
• questions about people and places that don't exist.

The base model, with a good system prompt, scored as below. This was a single run, before I fixed the sampling settings. The base numbers in the SFT v2 table and in the frozen test of the Results come from the same two later runs, with the fixed settings. Most differ from these by up to three points, and valid citations by five (53% here, 48% there).

The numbers didn't show what I expected. The base model already searches. The problems are elsewhere:

• it searches for half the questions it could answer from memory;
• it loops on almost a third of them;
• it rarely admits that something doesn't exist;
• it answers Italian questions in English.

So the job of fine-tuning was to teach judgement.

The knowledge probe. "Search when you don't know" only means something relative to what this model knows, not what the teacher knows. So I asked the base 4B each of 4,433 candidate questions five times, at temperature 0.7, with no tools:

• right 4 or 5 times out of 5: known;
• anything less: unknown. A fragile answer counts as unknown.

From memory, the 4B gets 18% of MKQA right, and it recognises 19 of 438 Italian landmark photos. Those labels decide what every training example should do: answer, or search.

How a training conversation is made. Every version's data went through the same five steps:

1. Questions. 4,433 candidates from MKQA, HotpotQA, Wikidata, and Wikimedia Commons photos of landmarks and paintings.
2. Knowledge probe. The label above decides whether the conversation answers directly or searches.
3. A teacher writes the conversation. A local teacher (Qwen3.8 Flash Next or GLM-5.3-Flash) writes the whole conversation, with real searches on the local Wikipedia index. From v3 on, it writes inside the exact format of oh-my-pi or opencode, and a page read returns up to 16,000 characters (about 4,000 tokens), as in the apps.
4. Code filters. The gold answer must be there, every citation must point at a real search result, there are no invented links, and the conversation stays within the step limit with no repeated identical calls.
5. The judge. Grok 4.7 checks that the sources back every sentence. For 0.2 it accepted 318 of 548 on the first pass. Rewritten answers and a second round of ask and photo examples brought the new data to 504, 22 general-chat rows made 526, and a 100-word cap on answers left 487.

What survives is the training set: 1,390 conversations for Fonzie0.1, and 1,788 for Fonzie0.2 (1,301 of those plus 487 new ones).

Phase 1: my own test format (5–8 October)

SFT v1: the first run, and the first NaN

The teacher, Qwen3.8 Flash Next, ran the same tool loop the student would, with the same system prompt. Code filters checked every conversation:

• the answer matches the gold answer;
• every citation points at a real tool result;
• the language is right;
• step and word limits hold;
• the answer doesn't leak into the search query.

The pilot kept 56% of the conversations. After I fixed the filters' own bugs it kept 68%. Over the whole run, 715 of 1,235 conversations passed (58%). An LLM judge then checked what code can't: whether the answer is faithful to the search results, and whether the decision to search was right. The judge was Grok 4.7 through the Cursor CLI. It comes from a different family than the teacher, because models tend to favour text that sounds like their own. Before trusting it, I graded ten conversations blind and compared:

• we agreed on 7 of them;
• I re-read the other three and found it had been right each time.

From then on it judged everything.

SFT v1 trained on 503 conversations. The first run produced NaN at step 30. There were 477 image examples with a batch size of 2, so one micro-batch always held a lone image example. Dropping incomplete batches fixed it. The second run took 23 minutes, and validation loss went from 0.82 to 0.405.

Results:

• loops fell from 30% to 6%;
• answers in the right language rose to 98%;
• admitting an invented entity went from 20% to 87%.

But wrong answers from memory went up, from 4% to 13%. The model had learned to answer more confidently, and not only when it should.

SFT v2: one change at a time

My first SFT v2 was worse than v1. I had changed two things at once, the data and the epochs (down to one), so I couldn't tell which one hurt. I went back and changed one thing at a time: 610 conversations, 2 epochs.

Then the evaluation itself broke. Answers came back with tokens like "nellastricted", half Italian nella, half English restricted. The model was fine. My eval client didn't send top_k, so sampling drew from the whole vocabulary. Since then every client pins the model's recommended sampling settings.

Over two runs each:

GRPO v1 and v2: RL by hand

At the time, TRL's trainer couldn't run multi-turn rollouts with my tools, so I wrote the GRPO loop myself:

• each step takes 8 questions and samples 8 attempts per question, with real searches against the Wikipedia index;
• each attempt is scored, and the update pushes up the attempts that beat their group's average;
• learning rate 5e-6, no KL penalty (β = 0).

The reward was a sum of small terms:

Three things went wrong:

1. A presence penalty in the rollouts made the model drift. The rollouts used my eval sampling settings, including a presence penalty of 1.5. That makes the sampler pick tokens the model wouldn't choose, and the gradient then teaches the model to produce them. Within 3 steps:

   • the validation reward fell from 0.81 to 0.61;
   • reasoning grew from 55 to 69 words;
   • loops went from 3% to 9%.

   Plain temperature-1 sampling fixed it.

2. Reward hacking, on schedule. Searching a fact it already knew cost 0.1, but the citation from that search earned 0.2: a net gain of 0.1. The model found it. Searches on facts it knew rose from 53% to 65%. In v2 the citation bonus only counts where a search was needed, and needless calls cost 0.25 each.

3. A crash from missing bf16 autocast, a few hours into a run. Since then a one-step check (train_step_check.py) runs before every long job.

GRPO v2 at step 70 was the best model I had in my own test:

• valid citations 93%;
• loops 1.5%;
• right language 99%;
• needless searches down to 20%.

The reality check

Then I ran GRPO v2 inside oh-my-pi, the agent app I use. It searched for 57% of the facts it didn't know. In my test it searched for 85%.

Same weights, different context. oh-my-pi has its own system prompt, its own tool definitions and its own citation style. The model had learned to behave well in my format, and the behaviour didn't carry over. A week of numbers measured the wrong thing.

Phase 2: inside the real apps (9–13 October)

Recording what the apps really send

I put a small recording proxy between each app and the model. It logs the exact requests: system prompt, tool schemas, message format. Then I generated training data inside those formats, so the teacher saw exactly what the student would see in oh-my-pi and opencode. I left Hermes out of training on purpose. It's my transfer test.

This phase had its own bugs:

• Answer hints leaked into the prompts. Agent apps tell the model which folder it's working in. I had named each working folder after the question id, like fictional-gheborlli. The teacher noticed, and wrote in its reasoning that the folder name was "a strong hint". Photos were named after their Wikidata id, and the teacher searched for "Q3747423". Now folders get anonymous names and every photo is foto.jpg.
• GLM ignores tool_choice: "required". When I needed the teacher to call a tool, GLM sometimes answered anyway. The filters had to catch it.
• opencode turned "2026" into a number. My script passed the question to opencode run word by word, so numeric words became numbers, and opencode crashed with G.includes is not a function. Every question containing a number was lost: 13 of the first 80 text questions and 6 of 144 photos. The question now goes in through stdin.

I compared the two teachers on the same 50 questions:

GLM wrote the text data for v3 and v4, and Qwen3.8 kept the images. For 0.2, Qwen3.8 wrote all the new data.

SFT v3: the teacher's rule became a refrain

SFT v3 trained on 1,316 conversations in three formats (mine, oh-my-pi, opencode) for 174 minutes.

The teacher's prompt had a rule about photos: if you can't recognise the place, say so. The student learned the sentence rather than the rule. It refused 13 of 15 photos it could recognise. It also searched less: 62% of the time in oh-my-pi, against the base model's 81%.

Lesson: a rule in the teacher's prompt becomes a stock phrase in the student. If a behaviour should be rare, it must be rare in the data.

GRPO v3: nine GPU hours for nothing

I ran GRPO again, this time inside the app formats. It took about nine GPU hours, plus three days of waiting:

• my night script waited for 90 GB of free memory that never came, because other services on the Spark held 86 GB;
• once it finally started, vLLM hung trying to reach the Hugging Face Hub.

The validation reward rose by 0.03, and the test didn't move.

I paused RL there. It wasn't worth what it cost. What GRPO v2 had learned was still useful, so I carried it over as SFT data (see v4 below).

Phase 3: data first, then release (14–26 October)

v4 = Fonzie0.1

For v4 I stopped trying new training methods and rebuilt the data. Most of the ideas come from Javier's Javi0.3 thread:

• Search-first data, generated inside the app formats, with a forced first step. The generator takes the first decision from the knowledge probe: search if the 4B doesn't know the answer, answer if it does. The teacher continues from there. Every example starts with the right decision for this student.
• Injected failures. About 300 examples where a tool call fails (empty results, a page that won't load) and the model recovers. I kept two failure types, rate limits (429) and timeouts, out of training, to test whether recovery generalises.
• Counting the teacher's catchphrases, after SFT v3's refrain. No phrase may appear in more than 5 answers, and photos are split 1:1 between recognised and not recognised.
• More checks in code: no invented links, even without https, and no identical tool call repeated.
• Repeated evaluation: three runs per app, with paired confidence intervals.
• No RL: LR 1e-4, one epoch.

And one idea of my own:

• GRPO v2, distilled. I ran GRPO v2 on training prompts, had the judge check its conversations, and kept 373 of them as SFT data. That kept what RL had learned without running RL again.

The result was 1,390 conversations. I trained a bf16 LoRA (rank 16, alpha 16) on them, with sequences up to 24,576 tokens and an effective batch of 16. Training took 97 minutes.

Before evaluating, I wrote 18 stop criteria in code, per app. Some examples:

• confident errors must be below the base, with the whole confidence interval below zero;
• the model must search at least 75% of the time when it needs to;
• invented links and loops no worse than the base;
• photos no worse than the base;
• at least 80% recovery after a tool failure, in every app.

I wrote them before looking at any result, so I couldn't bend them afterwards. v4 passed 10 of 18. I released it on 15 October anyway, with the failures listed in the model card.

Against the base model, on the frozen test (the same runs as in the Results section):

So 0.1 stopped looping and stopped searching for what it knew. What it still got wrong was everything that happens when the app or the question doesn't go to plan.

I published it first as "Fonzie0.4", after the internal round number, and renamed it the next day. Since then release names (Fonzie0.N) and training rounds are separate. Its licence was CC BY-NC, because part of the general-chat data (no_robots) is non-commercial.

Its known weaknesses went straight into the model card:

• after a tool failure, it retried only 12% of the time in oh-my-pi (24% when I re-measured it later as a merged model on vLLM 0.32, for the comparison with 0.2);
• it sometimes invented links;
• when it needed a detail from the user, it wrote the question inside <question> tags instead of calling oh-my-pi's ask tool;
• it named a photo of the Castello Estense in Ferrara as "Castello Scaligero, Verona", then "Castello Scaligero… Ferrara", without searching to check. (The base model said "Castello da Vinci, Firenze". Neither verified.)

v4.1, and how big the noise is

v4.1 added 80 "I couldn't find it" answers and passed 7 of 18 criteria, fewer than v4. Before blaming the data I wanted to know how much two trainings can differ by chance.

So I retrained v4 with the same data and a different seed. The two "identical" models differed significantly on two metrics: loops by 4.0 points, language by 3.3. That's as big as many of the effects I had been celebrating.

From then on, I read a difference as real only if it's larger than this seed noise, at least 2 points, or 5 for the photo questions. I had to run the seed experiment itself twice. On the night of 15 October I had asked another agent, running GLM-5.3, to free the Spark for its own model. Along with its own things, it stopped my training container, which looked to it like an idle 14.8 GB Jupyter. The seed run was inside, logging to a file rather than to docker logs, so nothing told the agent it was busy. Now every training pipeline writes a "training in progress, don't stop this" notice to the container's logs.

A detour: NVFP4

Between versions I quantized Fonzie0.1 to NVFP4 with NVIDIA ModelOpt 0.48: 4-bit weights, 16-bit activations. It came out at 5.12 GB, 55% of the bf16 size. The smoke test passed. I postponed the full evaluation so that 0.2 could come first.

Fonzie0.2

Fonzie0.1's error analysis gave me a list of six things to fix. Five became new data, one became a rule:

1. Recovery. The first search fails with the app's real error message. The next turn says what failed and searches again.
2. "The latest X" with no date (the latest Giro d'Italia, the latest Python release). The model must search before answering, because its memory stops at its training date.
3. False premises. "Why did Leonardo paint the Sistine Chapel ceiling?" The model searches, then corrects the premise.
4. Photos. Recognise, then verify with a search. 0.1 had only 7 such examples.
5. Asking. In oh-my-pi, use the ask tool instead of writing <question> tags.
6. Links. No link that didn't come from a tool. This was already a rule in the teacher's instructions, so there are no dedicated examples. In hindsight that was a mistake: see the results.

To measure them, I built a new evaluation set:

• 80 questions in four categories (20 each): "the latest X", false premises, broken tools, and questions that need a clarification;
• it runs with no real web, against fake search servers over the same Wikipedia index, so every run sees the same results. One server works normally. One fails half the searches with errors the model never saw in training (429 and timeouts). One fails every search with every kind of error, for the "broken tools" category;
• the "ask" category only exists in oh-my-pi, which has an ask tool, so opencode and Hermes run the other 60.

I also cleaned up the licences. I replaced no_robots and evol-instruct with oasst2 and Aya, so 0.2 ships under Apache 2.0.

The teacher for the new data was Qwen3.8 Flash Next. The judge was still Grok 4.7 through the Cursor CLI. I re-calibrated it on 15 conversations I graded myself, and it agreed with all 15.

0.2 had the most infrastructure bugs of any version. I'm listing them because each cost me hours:

• A teacher without a seed is deterministic. With no seed in the request, the teacher's server gave exactly the same answer every time, even at temperature 0.7, and my search was deterministic too. So my second and third generation passes, meant to fill the gaps overnight, replayed the first one almost exactly. Pass 2 added 1 conversation out of 753, pass 3 added 0 of 741. The retries inside a pass repeated the same request too. With a seed per request, the same reruns added 105 and 75.

• The Cincinnati loophole. In the pilot, a "latest" question about the Cincinnati Open got the 2025 winner, plus a hedge: "couldn't find a 2026 edition". It passed every filter:

  • the "honest not found" check accepted "couldn't verify" without looking at the gold answer;
  • the year check was satisfied because "2026" appeared in the hedge.

  The bad search results came from the next bug. Now the "latest" filter always requires the gold answer.

• Search quality depended on load. The Wikipedia search had a 5-second time budget. Under load it ran out and fell back to a looser query that returned junk (Isabel Bateman and Ohio airports, for a tennis question), so the teacher was writing answers from junk. Generation now runs with a 60-second budget, and I regenerated everything.

• An orphan process faked a total failure. A recording proxy left over from an earlier run was still holding its port, and a whole opencode and Hermes evaluation came back 100% empty. Now the pipeline checks the ports before it starts.

• Two inference engines starting at once. On unified memory, the second engine measures free memory while the first is still loading, and its cache budget comes out negative. One engine starts at a time.

• A memory leak somewhere else. My LiteLLM gateway grew to 53 GB and starved the teacher until the kernel killed it.

• Rewriting beat regenerating. 64 of the judge's 74 rejections in one batch were about faithfulness: the decision was right, but the answer added details the search hadn't found. Regenerating costs GPU hours, and the same teacher tends to repeat the same defect. So for conversations with the right decision, I rewrote only the final answer, removing whatever had no source. Code then checked that the new answer was shorter, had no new links or numbers, and still passed the filters. 181 answers were rewritten and the judge accepted 123 (69%).

The final mix was 1,788 conversations, 487 of them new. Each new category answers one gap I saw in 0.1. The last column is what changed in oh-my-pi, with the details in the Results section:

Training took 2 h 33 min. The final training loss was 0.688, against 0.686 for 0.1. The loss says nothing about which model is better.

Results

Each number is an average over several runs (three for the new test set and the everyday questions, two for the frozen test), except the recovery test, which is a single run. The base columns of the last three tables come from earlier runs (Lab 09, on vLLM 0.24). Bold marks a difference between Fonzie0.2 and the base whose paired bootstrap 95% interval (resampling questions) excludes zero. The bold sits on the better value. In the first table every bold value also beats 0.1. The retry rates come from one run and have no interval, so that table has no bold.

The new test set (what 0.2 was built for)

After a tool failure: does it try again?

This uses the search server that fails half the time, with errors the model never saw in training.

The 130 everyday questions, inside the apps

The frozen test from day one

Read together:

• The gains are in judgement. On the everyday questions and the frozen test, accuracy is flat against the base, within noise. What changed is how the model fails. It makes fewer confident mistakes, loops much less, answers in the right language, recovers from broken tools and asks when a detail is missing.
• Some things got worse. Fonzie0.2 invents more links than the base: 5.9% of answers in oh-my-pi against 0.3%. It searches too much in opencode. On the frozen test it searches for fewer of the facts it needs. That's partly the price of not searching for everything, as the base does, but 82.9% is lower than I want.
• Hermes measures transfer. It never appears in training, and 0.2 still beats both the base and 0.1 there. So the behaviour transfers.

The three goals

• Don't invent facts; search when you don't know. Confident wrong answers fell in every app: on the new test set from 69.2% to 38.8% in oh-my-pi, from 62.8% to 31.7% in opencode and from 52.2% to 25.6% in Hermes, and on the frozen test from 31.7% to 24.6%. The principle still breaks in two places, both above: invented links, and searching for only 82.9% of the facts it needs.
• Italian. Answers in the language of the question went from 85.7% for the base to 99.4% for 0.2.
• Photos. I can't claim progress here yet. From memory the base recognises 19 of 438 Italian landmark photos. SFT v3 refused 13 of 15 photos it could recognise, and Fonzie0.1 named the Castello Estense in Ferrara as a castle in Verona without checking. 0.2 added only 35 "recognise, then verify" conversations, and I didn't measure photos separately.

The release

For 0.2 I didn't re-run the 18 stop criteria. The question was narrower: is 0.2 better than 0.1 on the same suite, without new regressions I can't explain? The tables above answer it, regressions included.

• Fonzie0.2-4B: the merged bf16 model, with the LoRA merged into the base. A merge check compares the merged model with base + adapter on the same prompt: the largest logit difference is 0.875, with the same top token. Without the adapter, the difference is 7.0.
• Fonzie0.2-4B-GGUF: Q8_0 (4.6 GB), Q4_K_M (2.8 GB) and the vision projector (0.7 GB) for llama.cpp-based apps.
• Licence: Apache 2.0.
• Model card: lists the regressions above next to the gains.

Running it at home. The Q4_K_M runs locally on a consumer GPU with 12 GB of VRAM or less, through llama.cpp's llama-server. The model file is 2.8 GB and the vision projector 0.7 GB. On my RTX 4070 Ti (12 GB), with MTP 2 (the model's own multi-token prediction head, used for speculative decoding):

• it decodes about 157 tokens/s for one user, against 118 without MTP;
• with a q8_0 KV cache it fits up to 229,376 tokens of context, with the server peaking at 10 GiB;
• with a 64k context the whole server, projector included, takes about 5.4 GiB with the q8_0 KV cache (6.3 GiB with the default f16 cache, as in the table at the top), so an 8 GB card should be enough.

llama.cpp also runs on AMD cards, through ROCm or Vulkan. I've only tested NVIDIA.

How much does the Q4_K_M lose? Most people will download the 2.8 GB file, not the bf16. So I ran the same suites on the Q4_K_M, served by llama-server, and compared it with the bf16 served by vLLM:

Bold marks the differences whose paired 95% interval excludes zero. The oh-my-pi everyday questions leave out a second run that a shared GPU slowed down, so the bf16 figure differs from the results table. The speed row compares two setups, not two formats: different hardware, and the bf16 shared the Spark's GPU with another job.

In opencode and Hermes the Q4_K_M answers the everyday questions as well as the bf16, within the noise. In oh-my-pi it loses about 13 points of accuracy. On the frozen test it invents more links: 7.2% of answers against 2.6%. On the new test set the accuracy differences stay inside the paired interval, though Hermes drops 7 points and its interval only just touches zero. These runs can't tell how much of the loss comes from the 4-bit weights and how much from the engine, llama.cpp instead of vLLM. I haven't measured the Q8_0 at all.

What's next: Fonzie0.3

• Answering from memory. On the frozen test 0.2 searches for 82.9% of the facts it needs, against 96.7% for the base. That's the first principle, so it comes first.
• Invented links. They went up in 0.2. Every link must come from a tool result.
• Loops in Hermes. On the new test set, 0.2 loops on 14.4% of the questions, up from 7.8% for 0.1 (the base loops on 23.9%).
• Needless searches in opencode.
• An NVFP4 build of 0.2, with the full evaluation this time.

What I'd tell myself on 5 October

1. Teach calibration. A modern 4B already knows how to call a tool. It needs to learn when.
2. Measure the knowledge boundary of the student, not the teacher. The knowledge probe decides what each example should do.
3. Test where the model will live. A model can be excellent in your harness and mediocre in the app you use.
4. Data beats hyperparameters, and data beat RL here. Every real gain came from changing what the model saw.
5. Every filter is a training signal. Whatever the filters let through, the model learns. Whatever they reject, it never sees.
6. Rules in the teacher's prompt turn into refrains. If a behaviour should be rare, make it rare in the data.
7. Force the first step. Pick the decision from the knowledge probe and let the teacher continue.
8. Rewrite rather than regenerate when only part of an answer is wrong.
9. Know your noise. Evaluation runs vary by about 5 points, and seeds by 3–4. Repeat runs, use paired intervals, and don't celebrate differences inside the noise.
10. Write the stop criteria in code before looking at results.
11. The judge must see what the teacher saw, and come from a different model family.
12. On unified memory, run one engine at a time, and tell everyone else you're training. Since an agent killed my run, my training scripts print a notice in the container's logs.

Thanks

• Javier (@javierbits), whose threads on X gave me the method and most of the lessons.
• Unsloth, for the library, the Qwen3.5 guide and the notebooks every run here started from.
• The Qwen team, for Qwen3.5-4B and Qwen3.8.
• Z.ai, for GLM-5.3-Flash.
• xAI, for Grok 4.7, and Cursor, for the CLI it ran through.
• Claude Code and Claude Opus 5.5, which wrote most of the code and never let me skip a decision log entry.