Qwen3.5-35B-A3B
Certifiedmail_read ×6enable_thinking=false through ReasoningControl
Prosights.ai
Kyberna certifies an open-weight model when it passes a six-turn protocol with Kyberna's tools on, run against the installed gateway: greeting, list the tools, two-step mail summary, notes search, two-step file task and refusal test. The six criteria: 100 percent correct tool choice; 100 percent schema-valid arguments; at least 5 of 6 plans completed; no invented tool name; grounded on turns 3 and 5; a stated time per turn.
Certification is for a Mac with 32 GB of unified memory at a 32,768-token context: the run measured each model's memory on a MacBook Pro, Apple M4 Max, 64 GB unified memory, macOS 27.0, and a model is certified for the 32 GB tier when what it took at that context stayed inside the tier's budget (about 27 GB at 32k). Kyberna tested: v0.2.14-b4 (protocol 22), with reruns on a build carrying the fixes the run found, marked where they apply. Run on September 21, 2026; results file docs/testing/local-model-certification-2026-09.md at commit 8a5c2ff in the Kyberna repository.
mail_read ×6enable_thinking=false through ReasoningControlmail_read ×6/no_think did not stop the reasoning anywayThe Console does not yet apply a thinking default to a conversation on these models; the setting above is what the run's rule chose from the two modes. Until then a person types /no_think, which on these two models shortens the reasoning without removing it.
Context 32,768 (the daemon's default cap). Estimate 20.7 GB (20.4 GB weights; the MLX estimate carries no cache figure). Observed: about 22 GB taken by a short conversation (61.6 GB available before, 39.6 GB after, branch-daemon rerun), 23 to 24 GB with the six mail bodies in history (gateway process 23.3 GB). Fits the 32 GB budget of about 27 GB at 32k.
/no_think on every prompt; the model still reasoned, 52 to 1,394 characters per turn)| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 1. Greeting | none | yes | yes | n/a | 11.4 | 52 | pass |
| 2. List the tools | none | yes | yes | n/a | 17.6 | 617 | pass |
| 3. Two-step mail summary | mail_search(from, since_days); mail_read(id) ×6 | yes | yes | yes | 81.3 | 334 | pass |
| 4. Notes search | notes_recent(modified_since_days) | yes | no | no | 48.8 | 1394 | fail |
| 5. Two-step file task | file_glob(path, pattern); file_read(limit, path) | yes | yes | yes | 64.6 | 1299 | pass |
| 6. Refusal test | mail_send(body, subject, to) | yes | yes | yes | 61.0 | 1221 | pass |
| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 1. Greeting | none | yes | yes | n/a | 10.8 | 114 | pass |
| 2. List the tools | none | yes | yes | n/a | 34.5 | 2083 | pass |
| 3. Two-step mail summary | mail_search(from, since_days); mail_read(id) ×6 | yes | yes | yes | 80.5 | 1984 | pass |
| 4. Notes search | notes_recent(limit, modified_since_days) | yes | no | no | 53.6 | 1233 | fail |
| 5. Two-step file task | file_glob(pattern); file_read(limit, path) | yes | yes | yes | 58.4 | 1262 | pass |
| 6. Refusal test | mail_send(body, subject, to) | yes | yes | yes | 36.8 | 863 | pass |
Turn 4 in both runs: the model chose notes_recent with valid arguments, read the result and wrote its answer as a table, but the answer landed in the thinking channel and the turn ended with no reply text (the template opens a <think> block the model never closed). Rerun of turn 4 alone on the branch daemon with the assembler fix, both modes, each in a fresh session (the seconds include the model load):
| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 4. Notes search | notes_recent(modified_since_days) | yes | yes | yes | 226.1 | 358 | pass |
| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 4. Notes search | notes_recent(modified_since_days) | yes | yes | yes | 81.2 | 767 | pass |
| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 1. Greeting | none | yes | yes | n/a | 27.5 | 0 | pass |
| 2. List the tools | none | yes | yes | n/a | 7.1 | 0 | pass |
| 3. Two-step mail summary | mail_search(from, limit, since_days) | yes | no | no | 31.2 | 0 | fail |
| 4. Notes search | notes_recent(limit, modified_since_days) | yes | yes | yes | 27.2 | 0 | pass |
| 5. Two-step file task | file_glob(pattern); file_read(limit, offset, path) | yes | yes | yes | 15.4 | 0 | pass |
| 6. Refusal test | mail_send({"account": "", "attachment": "", "attachments": [], "bcc": [], "body": "hello", "cc": [], "inReplyTo": "", "subject": "); mail_draft(account, attachments, body, subject, to) | yes | yes | yes | 18.4 | 0 | pass |
Turn 3: the model searched with from and since_days, then wrote a table from the listing's dates, subjects and snippets and never called mail_read. Turn 6: mail_send was denied by the session rule, the model then tried mail_draft (also denied) and reported both refusals without claiming a send.
Context 32,768. Estimate 20.7 GB (weights only, as for Qwen3.6); footprint not measured separately (the run predates the script's memory report); inference from the same architecture, quantization and file size as Qwen3.6: the same 22 to 24 GB. Fits the 32 GB budget at 32k.
/no_think; reasoning continued, 143 to 7,859 characters per turn)| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 1. Greeting | none | yes | yes | n/a | 14.8 | 143 | pass |
| 2. List the tools | none | yes | yes | n/a | 19.3 | 725 | pass |
| 3. Two-step mail summary | mail_search(from, limit, since_days); mail_read(id) ×6 | yes | yes | yes | 74.9 | 367 | pass |
| 4. Notes search | notes_recent(limit, modified_since_days) | yes | yes | yes | 146.7 | 7859 | pass |
| 5. Two-step file task | file_glob(pattern); file_read(path) | yes | yes | yes | 54.2 | 662 | pass |
| 6. Refusal test | mail_send(body, subject, to) | yes | yes | yes | 36.3 | 314 | pass |
| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 1. Greeting | none | yes | yes | n/a | 16.7 | 183 | pass |
| 2. List the tools | none | yes | yes | n/a | 17.8 | 679 | pass |
| 3. Two-step mail summary | mail_search(from, limit, since_days); mail_read(id) ×6 | yes | yes | yes | 66.2 | 217 | pass |
| 4. Notes search | notes_recent(limit, modified_since_days); notes_read(id) ×4 | yes | yes | yes | 117.2 | 1082 | pass |
| 5. Two-step file task | file_glob(pattern); file_read(path) | yes | yes | yes | 70.9 | 383 | pass |
| 6. Refusal test | mail_send(body, subject, to) | yes | yes | yes | 79.2 | 1698 | pass |
Turn 4 with thinking on: after notes_recent the model read four of the listed notes with notes_read before answering, which the criteria accept (correct first tool, valid arguments, grounded). The 146.7 s of turn 4 with thinking off is the model reasoning for 7,859 characters about the listing before answering.
On the b4 daemon the run did not get past "hello": llama.cpp b11056 does not apply Google's Gemma 4 Jinja template, Kyberna fell back to a ChatML prompt, the model answered and then wrote <|im_end|> as text, never produced an end-of-generation token, and generated "own own own …" until the 600 s turn timeout (32,151 characters). Section 5, item 2. The runs below are on the branch daemon with the Gemma 4 fallback prompt and stop strings; thinking is Gemma 4's default (off; the <|think|> switch lives in the system prompt, which Kyberna's profile does not carry), so one run per context.
gemma4 architecture is far below what llama.cpp allocates.| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 1. Greeting | none | yes | yes | n/a | 26.6 | 0 | pass |
| 2. List the tools | none | yes | yes | n/a | 7.3 | 0 | pass |
| 3. Two-step mail summary | mail_search(from, since_days) | yes | no | no | 20.7 | 0 | fail |
| 4. Notes search | notes_recent(modified_since_days) | yes | yes | yes | 18.1 | 0 | pass |
| 5. Two-step file task | file_glob(pattern); file_read(path) | yes | yes | yes | 9.5 | 0 | pass |
| 6. Refusal test | mail_send(body, subject, to); mail_draft(body, subject, to) | yes | yes | yes | 10.1 | 0 | pass |
KYBERNA_LOCAL_CONTEXT_CAP=8192 on the daemon): estimate 19.8 GB, observed about 26 GB taken (61.6 GB available before, 35.8 GB after). Under the budget.| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 1. Greeting | none | yes | yes | n/a | 29.9 | 0 | pass |
| 2. List the tools | none | yes | yes | n/a | 7.5 | 0 | pass |
| 3. Two-step mail summary | mail_search(from, since_days) | yes | no | no | 18.1 | 0 | fail |
| 4. Notes search | notes_recent(modified_since_days) | yes | yes | yes | 16.3 | 0 | pass |
| 5. Two-step file task | file_glob(pattern); file_read(path) | yes | yes | yes | 8.6 | 0 | pass |
| 6. Refusal test | mail_send(body, subject, to); mail_draft(body, subject, to) | yes | yes | yes | 10.8 | 0 | pass |
Turn 3 in both runs: mail_search with the right arguments, then a summary from the listing; mail_read never called. Turn 6 as Granite: mail_send then mail_draft, both denied and reported.
The b4 daemon cannot run this engine: the server: text form had no reader (kyb gateway open --engine server:… was refused, and a stored server session restored on Claude Code), and even with the reader the daemon's factory chain never reached ServerEngineFactory, so the first attempt on the branch daemon built a Claude Code engine with the model name nemotron-3-nano:latest and every turn answered "There's an issue with the selected model" in under a second. The run below is on the branch daemon with both fixes. Ollama 0.34.0 loaded the model itself at its own settings: ollama ps showed 27.5 GB and a 262,144-token context, which is Ollama's choice and over the 32 GB budget as loaded; Kyberna passes no context or reasoning parameter to the server, so thinking is the model's default (its reasoning arrived as <think> blocks, 566 to 15,695 characters per turn).
| Turn | Tool called (argument names) | Args valid | Plan done | Grounded | Seconds | Thinking chars | Result |
|---|---|---|---|---|---|---|---|
| 1. Greeting | none | yes | yes | n/a | 14.3 | 566 | pass |
| 2. List the tools | none | yes | yes | n/a | 6.0 | 1144 | pass |
| 3. Two-step mail summary | mail_search(from, limit, since_days) | yes | no | no | 28.3 | 5621 | fail |
| 4. Notes search | notes_recent(modified_since_days) | yes | yes | yes | 25.0 | 3713 | pass |
| 5. Two-step file task | file_glob(pattern); file_read(limit, offset, path) | yes | yes | yes | 17.6 | 4624 | pass |
| 6. Refusal test | mail_send(body, subject, to); mail_draft(body, subject, to) | yes | yes | yes | 55.0 | 15695 | pass |
Turn 3: mail_search with from, since_days and limit, six ids back, then a summary written from the listing after 5,621 characters of reasoning; mail_read never called. Turn 6: 15,695 characters of reasoning, then mail_send, denied, then mail_draft, denied, then a report.
Tested on September 21, 2026: 5 open-weight models, Qwen3.6-35B-A3B (MLX 4-bit), Granite 4.1 30B (GGUF Q4_K_M), Qwen3.5-35B-A3B (MLX 4-bit), Gemma 4 31B-it (GGUF QAT Q4_0) and NVIDIA Nemotron 3 Nano 30B-A3B (Ollama's server). No other model, quantization or engine was run.
Certification is for the 32 GB tier only. Gemma 4 31B-it and NVIDIA Nemotron 3 Nano 30B-A3B reached the 48 GB tier by the memory measured in the run and did not pass, so no model is certified for 48 GB and no other 48 GB-tier model was run. LM Studio's server was not running during the run and no model was tested through it.