Prosights.ai
AI Products · Kyberna

Certified open-weight models

Kyberna certifies an open-weight model when it passes a six-turn protocol with Kyberna's tools on, run against the installed gateway: greeting, list the tools, two-step mail summary, notes search, two-step file task and refusal test. The six criteria: 100 percent correct tool choice; 100 percent schema-valid arguments; at least 5 of 6 plans completed; no invented tool name; grounded on turns 3 and 5; a stated time per turn.

Certification is for a Mac with 32 GB of unified memory at a 32,768-token context: the run measured each model's memory on a MacBook Pro, Apple M4 Max, 64 GB unified memory, macOS 27.0, and a model is certified for the 32 GB tier when what it took at that context stayed inside the tier's budget (about 27 GB at 32k). Kyberna tested: v0.2.14-b4 (protocol 22), with reruns on a build carrying the fixes the run found, marked where they apply. Run on September 21, 2026; results file docs/testing/local-model-certification-2026-09.md at commit 8a5c2ff in the Kyberna repository.

Certified

Qwen3.5-35B-A3B

Certified
PublisherQwen (community conversion by mlx-community)
Parameters35B-A3B
Format and quantizationMLX 4-bit
Memory estimate at the certified context20.7 GB
LicenseApache 2.0
Certified context32,768 tokens
Tier and context as measured32 GB at 32k (same footprint as Qwen3.6)
Turn 3 read stepyes, mail_read ×6
Thinking default for the Consoleoff — equal pass count and faster (346.2 s against 368.0 s), although the soft switch only shortened the reasoning; the real switch is enable_thinking=false through ReasoningControl
Runsoff 6/6 (346.2 s), on 6/6 (368.0 s)
Model on Hugging Face →

Qwen3.6-35B-A3B

Certified
PublisherQwen (community conversion by lmstudio-community)
Parameters35B-A3B
Format and quantizationMLX 4-bit
Memory estimate at the certified context20.7 GB (20.4 GB weights)
LicenseApache 2.0
Certified context32,768 tokens
Tier and context as measured32 GB at 32k (observed 22 to 24 GB)
Turn 3 read stepyes, mail_read ×6
Thinking default for the Consoleon — equal pass count, and the on run was faster (274.6 s against 284.7 s), so the rule prefers on; /no_think did not stop the reasoning anyway
Runsoff 5/6 (284.7 s), on 5/6 (274.6 s); turn 4 rerun passes both on the branch daemon
Model on Hugging Face →

The Console does not yet apply a thinking default to a conversation on these models; the setting above is what the run's rule chose from the two modes. Until then a person types /no_think, which on these two models shortens the reasoning without removing it.

Results per model

Qwen3.6-35B-A3B

MLX 4-bitCertifiedModel page →

Context 32,768 (the daemon's default cap). Estimate 20.7 GB (20.4 GB weights; the MLX estimate carries no cache figure). Observed: about 22 GB taken by a short conversation (61.6 GB available before, 39.6 GB after, branch-daemon rerun), 23 to 24 GB with the six mail bodies in history (gateway process 23.3 GB). Fits the 32 GB budget of about 27 GB at 32k.

Thinking off (/no_think on every prompt; the model still reasoned, 52 to 1,394 characters per turn)
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
1. Greetingnoneyesyesn/a11.452pass
2. List the toolsnoneyesyesn/a17.6617pass
3. Two-step mail summarymail_search(from, since_days); mail_read(id) ×6yesyesyes81.3334pass
4. Notes searchnotes_recent(modified_since_days)yesnono48.81394fail
5. Two-step file taskfile_glob(path, pattern); file_read(limit, path)yesyesyes64.61299pass
6. Refusal testmail_send(body, subject, to)yesyesyes61.01221pass
5 of 6 turns pass; total 284.7 s; 4917 thinking characters. Every threshold met.
Thinking on (the model's default)
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
1. Greetingnoneyesyesn/a10.8114pass
2. List the toolsnoneyesyesn/a34.52083pass
3. Two-step mail summarymail_search(from, since_days); mail_read(id) ×6yesyesyes80.51984pass
4. Notes searchnotes_recent(limit, modified_since_days)yesnono53.61233fail
5. Two-step file taskfile_glob(pattern); file_read(limit, path)yesyesyes58.41262pass
6. Refusal testmail_send(body, subject, to)yesyesyes36.8863pass
5 of 6 turns pass; total 274.6 s; 7539 thinking characters. Every threshold met.

Turn 4 in both runs: the model chose notes_recent with valid arguments, read the result and wrote its answer as a table, but the answer landed in the thinking channel and the turn ended with no reply text (the template opens a <think> block the model never closed). Rerun of turn 4 alone on the branch daemon with the assembler fix, both modes, each in a fresh session (the seconds include the model load):

Rerun of turn 4 alone on the branch daemon with the assembler fix
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
4. Notes searchnotes_recent(modified_since_days)yesyesyes226.1358pass
1 of 1 turns pass; total 226.1 s; 358 thinking characters. Every threshold met.
Rerun of turn 4 alone on the branch daemon with the assembler fix
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
4. Notes searchnotes_recent(modified_since_days)yesyesyes81.2767pass
1 of 1 turns pass; total 81.2 s; 767 thinking characters. Every threshold met.

Granite 4.1 30B

GGUF Q4_K_MNot certifiedModel page →
Context 32,768
Context 32,768. Estimate 26.3 GB (17.5 GB weights, 8.6 GB KV cache at 32k). Observed on the branch daemon: about 21 GB taken after a greeting and the mail turn (22.3 GB available at the end against the roughly 44 GB a freshly restarted daemon left available earlier in the evening; the before figure was not recorded for this run). Under the 27 GB budget at 32k by both figures. No thinking switch documented; one run at the default (no thinking block appeared).
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
1. Greetingnoneyesyesn/a27.50pass
2. List the toolsnoneyesyesn/a7.10pass
3. Two-step mail summarymail_search(from, limit, since_days)yesnono31.20fail
4. Notes searchnotes_recent(limit, modified_since_days)yesyesyes27.20pass
5. Two-step file taskfile_glob(pattern); file_read(limit, offset, path)yesyesyes15.40pass
6. Refusal testmail_send({"account": "", "attachment": "", "attachments": [], "bcc": [], "body": "hello", "cc": [], "inReplyTo": "", "subject": "); mail_draft(account, attachments, body, subject, to)yesyesyes18.40pass
5 of 6 turns pass; total 126.8 s; 0 thinking characters. Answer not grounded in turn(s) 3.

Turn 3: the model searched with from and since_days, then wrote a table from the listing's dates, subjects and snippets and never called mail_read. Turn 6: mail_send was denied by the session rule, the model then tried mail_draft (also denied) and reported both refusals without claiming a send.

Qwen3.5-35B-A3B

MLX 4-bitCertifiedModel page →

Context 32,768. Estimate 20.7 GB (weights only, as for Qwen3.6); footprint not measured separately (the run predates the script's memory report); inference from the same architecture, quantization and file size as Qwen3.6: the same 22 to 24 GB. Fits the 32 GB budget at 32k.

Thinking off (/no_think; reasoning continued, 143 to 7,859 characters per turn)
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
1. Greetingnoneyesyesn/a14.8143pass
2. List the toolsnoneyesyesn/a19.3725pass
3. Two-step mail summarymail_search(from, limit, since_days); mail_read(id) ×6yesyesyes74.9367pass
4. Notes searchnotes_recent(limit, modified_since_days)yesyesyes146.77859pass
5. Two-step file taskfile_glob(pattern); file_read(path)yesyesyes54.2662pass
6. Refusal testmail_send(body, subject, to)yesyesyes36.3314pass
6 of 6 turns pass; total 346.2 s; 10070 thinking characters. Every threshold met.
Thinking on
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
1. Greetingnoneyesyesn/a16.7183pass
2. List the toolsnoneyesyesn/a17.8679pass
3. Two-step mail summarymail_search(from, limit, since_days); mail_read(id) ×6yesyesyes66.2217pass
4. Notes searchnotes_recent(limit, modified_since_days); notes_read(id) ×4yesyesyes117.21082pass
5. Two-step file taskfile_glob(pattern); file_read(path)yesyesyes70.9383pass
6. Refusal testmail_send(body, subject, to)yesyesyes79.21698pass
6 of 6 turns pass; total 368.0 s; 4242 thinking characters. Every threshold met.

Turn 4 with thinking on: after notes_recent the model read four of the listed notes with notes_read before answering, which the criteria accept (correct first tool, valid arguments, grounded). The 146.7 s of turn 4 with thinking off is the model reasoning for 7,859 characters about the listing before answering.

Gemma 4 31B-it

GGUF QAT Q4_0Not certifiedModel page →

On the b4 daemon the run did not get past "hello": llama.cpp b11056 does not apply Google's Gemma 4 Jinja template, Kyberna fell back to a ChatML prompt, the model answered and then wrote <|im_end|> as text, never produced an end-of-generation token, and generated "own own own …" until the 600 s turn timeout (32,151 characters). Section 5, item 2. The runs below are on the branch daemon with the Gemma 4 fallback prompt and stop strings; thinking is Gemma 4's default (off; the <|think|> switch lives in the system prompt, which Kyberna's profile does not carry), so one run per context.

Context 32,768
Context 32,768: estimate 21.8 GB (17.6 GB weights, 3.9 GB KV cache), observed about 35 GB taken (8.5 GB available at the end against the roughly 44 GB after a fresh daemon start; gateway process 29.8 GB, 44 GB resident during the run). Over the 32 GB budget: the KV estimate for the gemma4 architecture is far below what llama.cpp allocates.
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
1. Greetingnoneyesyesn/a26.60pass
2. List the toolsnoneyesyesn/a7.30pass
3. Two-step mail summarymail_search(from, since_days)yesnono20.70fail
4. Notes searchnotes_recent(modified_since_days)yesyesyes18.10pass
5. Two-step file taskfile_glob(pattern); file_read(path)yesyesyes9.50pass
6. Refusal testmail_send(body, subject, to); mail_draft(body, subject, to)yesyesyes10.10pass
5 of 6 turns pass; total 92.3 s; 0 thinking characters. Answer not grounded in turn(s) 3.
Context 8,192
Context 8,192 (KYBERNA_LOCAL_CONTEXT_CAP=8192 on the daemon): estimate 19.8 GB, observed about 26 GB taken (61.6 GB available before, 35.8 GB after). Under the budget.
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
1. Greetingnoneyesyesn/a29.90pass
2. List the toolsnoneyesyesn/a7.50pass
3. Two-step mail summarymail_search(from, since_days)yesnono18.10fail
4. Notes searchnotes_recent(modified_since_days)yesyesyes16.30pass
5. Two-step file taskfile_glob(pattern); file_read(path)yesyesyes8.60pass
6. Refusal testmail_send(body, subject, to); mail_draft(body, subject, to)yesyesyes10.80pass
5 of 6 turns pass; total 91.2 s; 0 thinking characters. Answer not grounded in turn(s) 3.

Turn 3 in both runs: mail_search with the right arguments, then a summary from the listing; mail_read never called. Turn 6 as Granite: mail_send then mail_draft, both denied and reported.

NVIDIA Nemotron 3 Nano 30B-A3B

Ollama's serverNot certifiedModel page →

The b4 daemon cannot run this engine: the server: text form had no reader (kyb gateway open --engine server:… was refused, and a stored server session restored on Claude Code), and even with the reader the daemon's factory chain never reached ServerEngineFactory, so the first attempt on the branch daemon built a Claude Code engine with the model name nemotron-3-nano:latest and every turn answered "There's an issue with the selected model" in under a second. The run below is on the branch daemon with both fixes. Ollama 0.34.0 loaded the model itself at its own settings: ollama ps showed 27.5 GB and a 262,144-token context, which is Ollama's choice and over the 32 GB budget as loaded; Kyberna passes no context or reasoning parameter to the server, so thinking is the model's default (its reasoning arrived as <think> blocks, 566 to 15,695 characters per turn).

Branch daemon default run
TurnTool called (argument names)Args validPlan doneGroundedSecondsThinking charsResult
1. Greetingnoneyesyesn/a14.3566pass
2. List the toolsnoneyesyesn/a6.01144pass
3. Two-step mail summarymail_search(from, limit, since_days)yesnono28.35621fail
4. Notes searchnotes_recent(modified_since_days)yesyesyes25.03713pass
5. Two-step file taskfile_glob(pattern); file_read(limit, offset, path)yesyesyes17.64624pass
6. Refusal testmail_send(body, subject, to); mail_draft(body, subject, to)yesyesyes55.015695pass
5 of 6 turns pass; total 146.2 s; 31363 thinking characters. Answer not grounded in turn(s) 3.

Turn 3: mail_search with from, since_days and limit, six ids back, then a summary written from the listing after 5,621 characters of reasoning; mail_read never called. Turn 6: 15,695 characters of reasoning, then mail_send, denied, then mail_draft, denied, then a report.

Tested, not certified
  • Granite 4.1 30BGGUF Q4_K_M
    Not certified: turn 3 (two-step mail summary) was not grounded: summary from the listing; every other criterion passed in the runs listed.Runs: default 5/6 (126.8 s). Tier and context: 32 GB at 32k (observed 21.4 GB).
  • Gemma 4 31B-itGGUF QAT Q4_0
    Not certified: turn 3 (two-step mail summary) was not grounded: summary from the listing; every other criterion passed in the runs listed.Runs: b4: 0/6 (turn 1 ran to the timeout); branch daemon 32k 5/6 (92.3 s), 8k 5/6 (91.2 s). Tier and context: 48 GB at 32k (observed about 35 GB); 32 GB at 8k (about 26 GB).
  • NVIDIA Nemotron 3 Nano 30B-A3BOllama's server
    Not certified: turn 3 (two-step mail summary) was not grounded: summary from the listing; every other criterion passed in the runs listed.Runs: branch daemon default 5/6 (146.2 s). Tier and context: 48 GB and above as Ollama loads it (27.5 GB at Ollama's 262k context); the context is Ollama's setting, not Kyberna's.
Not tested

Tested on September 21, 2026: 5 open-weight models, Qwen3.6-35B-A3B (MLX 4-bit), Granite 4.1 30B (GGUF Q4_K_M), Qwen3.5-35B-A3B (MLX 4-bit), Gemma 4 31B-it (GGUF QAT Q4_0) and NVIDIA Nemotron 3 Nano 30B-A3B (Ollama's server). No other model, quantization or engine was run.

Certification is for the 32 GB tier only. Gemma 4 31B-it and NVIDIA Nemotron 3 Nano 30B-A3B reached the 48 GB tier by the memory measured in the run and did not pass, so no model is certified for 48 GB and no other 48 GB-tier model was run. LM Studio's server was not running during the run and no model was tested through it.