Prosights.ai
← Features
AI Products · Kyberna · Features

The psychology, behavioral science, and AI research behind Kyberna

54 published papers either shaped or support how Kyberna works.

Used in the design means Kyberna used the paper when it designed the feature.

Agrees with how it works means a paper found later supports how the feature works.

Cuts against how it works means a paper contradicts how the feature works.

"I Need to Find That One Chart": How Data Workers Navigate, Make Sense of, and Communicate Analytical Conversations

Interfaces limited to linear scrolling and keyword search constrained people returning to past conversations, and participants used filtering and multi-level navigation. 10 of 10 participants needed to reorient, 4 of 10 needed to recall specific items, and 8 of 10 used the detail explorer. The paper is a qualitative study of 10 people and gives no effect size.

Authors: Ken Gu (University of Washington), Srishti Palani (Tableau Research), Vidya Setlur (Tableau Research)

Published in: CHI 2026

How the study was done: A user study with a design probe that added structure to past conversations. 10 data workers.

Cuts against how it works: Find in transcript. It searches keywords over a linear transcript. This study found that form limiting, for 10 data workers.

Read the paper

A Mixed-Methods Study on User Experiences and Challenges of Recovery Codes for an End-to-End Encrypted Service

Most participants stored the recovery code, and a password manager was the most common place of 6 strategies.

Authors: Sandra Höltervennhoff (Leibniz University Hannover), Noah Wöhler (CISPA), and colleagues

Published in: USENIX Security 2024

How the study was done: An online survey of an encrypted email service's users and an analysis of support requests on Reddit. 281 survey participants and 196 support requests.

Used in the design: Recovery key.

Read the paper

AgentDojo

Attacks broke some security properties, and agents failed tasks even without an attack. Agents solved under 66% of tasks with no attack; attacks succeeded under 25% of the time against the best agents; a tool filter brought that to 7.5%.

Authors: Edoardo Debenedetti, Jie Zhang (ETH Zurich), Mislav Balunović, Luca Beurer-Kellner, Marc Fischer (ETH Zurich, Invariant Labs), Florian Tramèr (ETH Zurich)

Published in: NeurIPS 2024 Datasets and Benchmarks

How the study was done: A framework that runs agents on tool tasks with prompt injection attacks and defenses. 97 tasks and 629 security cases.

Used in the design: Folder Boundary, Permission Boundary.

Read the paper

AgentHarm

Leading models complied with many malicious agent requests without any jailbreak. Without a jailbreak, Mistral Large 2 scored 82.2% harm with 1.1% refusals; Claude 3.5 Sonnet scored 13.5% with 85.2% refusals.

Authors: Maksym Andriushchenko, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson (Gray Swan AI), Alexandra Souly, Eric Winsor, Jerome Wynne, Yarin Gal, Xander Davies (UK AI Security Institute)

Published in: ICLR 2025

How the study was done: A benchmark of malicious multi-step agent tasks across leading models. 110 tasks, 440 with augmentations, across 11 harm categories.

Used in the design: Tool policy, Folder Boundary, Folder Snapshots, Permission Boundary.

Read the paper

Alice in Warningland: A Large-Scale Field Study of Browser Security Warning Effectiveness

Users continued through a tenth of Firefox's malware and phishing warnings, a quarter of Chrome's, a third of Firefox's SSL warnings and 70.2% of Chrome's SSL warnings.

Authors: Devdatta Akhawe (University of California, Berkeley), Adrienne Porter Felt (Google)

Published in: USENIX Security 2013

How the study was done: A field study from Firefox and Chrome telemetry. Over 25 million warning impressions.

Used in the design: Kyberna Gateway, Composer, Readable transcript, Projects, Folder switching, Loaded Models, Memory check, Browser use in conversations, Kyberna Guard, Approval requests, Approval levels, Remembered answers, Site approvals, Loop detection, Rogue Agent Protection, On-device reminder routing, Folder Boundary, Reminder file access, Paused background runs, Explain This Request, Agents a conversation starts, Folder Snapshots, Messages between conversations, Web pages Kyberna's own engines read, Permission Boundary, Decide for me.

Read the paper

Android Permissions Remystified: A Field Study on Contextual Integrity

At least 80% of participants would have prevented at least 1 permission request, and overall they wanted to block over a third of all requests.

Authors: Primal Wijesekera (University of British Columbia), Arjun Baokar, Ashkan Hosseini, Serge Egelman, David Wagner (University of California, Berkeley), Konstantin Beznosov (University of British Columbia)

Published in: USENIX Security 2015

How the study was done: A field study on instrumented Android phones with exit interviews. 36 participants for 1 week, 27 million data points.

Used in the design: Kyberna Gateway, Kyberna tools switch, Mail tools, Kyberna Guard, Approval requests, Remembered answers, Site approvals, Pause All Activity, Rogue Agent Protection, Reminder file access, Commands run by local and API models, Explain This Request, Permission Boundary.

Read the paper

Android Permissions: User Attention, Comprehension, and Behavior

17% of participants paid attention to permissions during installation, and 3% of survey respondents answered all 3 comprehension questions correctly.

Authors: Adrienne Porter Felt, Elizabeth Ha, Serge Egelman, Ariel Haney, Erika Chin, David Wagner (University of California, Berkeley)

Published in: SOUPS 2012

How the study was done: An Internet survey and a laboratory study of Android users installing apps. 308 survey respondents and 25 laboratory participants.

Used in the design: Kyberna Gateway, Projects, Folder switching, Model servers, Kyberna tools switch, Kyberna Guard, Approval requests, Remembered answers, Package Sites, Site approvals, Pause All Activity, Rogue Agent Protection, Folder Boundary, Reminder file access, Turn status line, Paused background runs, Commands run by local and API models, Send checks, Explain This Request, Background tasks list, Agents a conversation starts, Serve Kyberna’s local models to other Macs and devices, Messages between conversations, Permission Boundary, Ask About This Request, Decide for me.

Read the paper

ASB (Agent Security Bench)

The highest average attack success was 84.30%, and the defenses had limited effect.

Authors: Hanrong Zhang and colleagues (Zhejiang University, Rutgers University)

Published in: ICLR 2025

How the study was done: A benchmark of attacks and defenses on agents. 10 scenarios, over 400 tools, 13 backbones and 11 defenses.

Used in the design: Folder Boundary, Permission Boundary.

Read the paper

Breaking Agents

Failure rates passed 80% in several scenarios, against a baseline malfunction rate of 15.3%.

Authors: Boyang Zhang, Yicong Tan (CISPA), Yun Shen (Flexera), Ahmed Salem (Microsoft), Michael Backes (CISPA), Savvas Zannettou (TU Delft), Yang Zhang (CISPA)

Published in: EMNLP 2025

How the study was done: Attacks that push agents into repetitive or irrelevant actions. Agents across several scenarios.

Used in the design: Folder Boundary, Permission Boundary.

Read the paper

Can Systems Explain Permissions Better?

6.1% of users could infer the scope of permission groups accurately from what the system showed.

Authors: Bingyu Shen (University of California, San Diego), Lili Wei (The Hong Kong University of Science and Technology), and colleagues

Published in: USENIX Security 2021

How the study was done: Mixed methods: real permission settings, an interview study and 2 online surveys. 4,636 Android users' settings, 20 interviews, surveys of 359 and 1,200.

Used in the design: Kyberna Gateway.

Read the paper

Co-Writing with Opinionated Language Models Affects Users' Views

The assistant changed the opinions people expressed in their writing and shifted their later survey answers. Users of the opinionated assistant were 2.04 times more likely to write the same view (95% CI 1.83-2.30, p<0.0001); opinion moved by about 0.29, d=0.5.

Authors: Maurice Jakesch (Cornell University), Advait Bhat (Microsoft Research India), Daniel Buschek (University of Bayreuth), Lior Zalmanson (Tel Aviv University), Mor Naaman (Cornell Tech)

Published in: CHI 2023

How the study was done: An online experiment in which some participants wrote with an assistant configured to argue one side. 1,506 participants.

Cuts against how it works: Writing actions. Each result waits for you to accept it. This study found accepted suggestions can still move opinions, with an assistant set to argue 1 side.

Read the paper

Comparing Speech and Keyboard Text Entry for Short Messages in Two Languages on Touchscreen Phones

Speech was 2.93 times faster than the keyboard in English (153 against 52 words per minute).

Authors: Sherry Ruan (Stanford University), Jacob O. Wobbrock (University of Washington), Kenny Liou (Symantec), Andrew Ng (Stanford University), James A. Landay (Stanford University)

Published in: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1(4), 2017

How the study was done: A laboratory experiment on an iPhone 6 Plus comparing a speech recognizer with the built-in keyboards. Short message transcription in English and Mandarin Chinese.

Agrees with how it works: Dictation. Dictation lets you speak instead of type. The study timed phone touchscreens, not a Mac keyboard.

Read the paper

Compressing Context to Enhance Inference Efficiency of Large Language Models

A 50% reduction in context cut memory 36% and time 32%, with drops of .023 in BERTScore and .038 in faithfulness. The paper reports no standardized effect size.

Authors: Yucheng Li, Bo Dong, Frank Guerin (University of Surrey), Chenghua Lin (University of Manchester)

Published in: EMNLP 2023

How the study was done: Drops low self-information words from the context and compares the answers with those from the full context. BBC News, arXiv papers and ShareGPT conversations; 4 tasks; GPT-3.5, GPT-4, LLaMA and Vicuna.

Used in the design: Token Optimization.

Read the paper

Dark Patterns after the GDPR: Scraping Consent Pop-ups and Demonstrating their Influence

Removing the opt-out button from the first page raised consent by 22 to 23 percentage points, and 11.8% of sites met the minimal legal requirements set by the authors.

Authors: Midas Nouwens (Aarhus University, MIT CSAIL), Ilaria Liccardi, David Karger, Lalana Kagal (MIT CSAIL), Michael Veale (University College London)

Published in: CHI 2020

How the study was done: A scrape of consent pop-ups on UK web sites and a field experiment on 8 common designs. 680 web sites and 40 participants.

Agrees with how it works: Cookie banners. Kyberna declines for you. This study shows banner design moves people toward accepting, on UK web sites.

Read the paper

Designing Usable Web Forms – Empirical Evaluation of Web Form Improvement Guidelines

Improved forms led to faster completion, fewer submission trials, fewer eye movements and higher satisfaction. Completion time improved with d=1.00 and d=.93 on 2 forms; the third, d=.38, was not significant.

Authors: Mirjam Seckler, Silvia Heinz, Klaus Opwis, Alexandre N. Tuch (University of Basel), Javier A. Bargas-Avila (Google)

Published in: CHI 2014

How the study was done: A controlled eye tracking experiment on company web forms, with and without 20 guidelines applied. 65 participants.

Used in the design: Setup sheet, Recovery key, Model servers, Local models, Encrypted Record, Turn status line, Send checks, Move to a new Mac, Agents a conversation starts, Serve Kyberna’s local models to other Macs and devices, Messages between conversations.

Read the paper

Do Defaults Save Lives?

Making donation the default raised donation by 16.3% in the country analysis (P < 0.02).

Authors: Eric J. Johnson (Columbia University), Daniel G. Goldstein (Columbia University)

Published in: Science 302, 2003

How the study was done: Country data on organ donation and an online experiment varying the default. 161 respondents in the online experiment.

Used in the design: Setup sheet, Recovery key, Local models, Context and thinking, Send allowlist, Encrypted Record, Pause and Kill a Session, Pause All Activity, Paused background runs, Commands run by local and API models, Background tasks list.

Read the paper

Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making

A per-case confidence score helped people calibrate their trust in the model. Confidence had a main effect on trust, F(1,64)=4.64, p=.035; the calibration interaction was F(4,256)=15.8, p<.001, partial eta squared .198 (computed).

Authors: Yunfeng Zhang, Q. Vera Liao, Rachel K. E. Bellamy (IBM Research AI)

Published in: ACM FAT* 2020

How the study was done: An online experiment on AI-assisted decisions, showing confidence or explanations. 72 participants on Amazon Mechanical Turk in the first experiment.

Used in the design: Pause All Activity, Rogue Agent Protection.

Read the paper

Effects of Intelligent Notification Management on Users and Their Tasks

Scheduling notifications at breakpoints reduced frustration and reaction time against delivering them at once. Frustration differed by policy, F(3,109)=5.4, p<0.002, partial eta squared .129 (computed); reaction time, F(1,66)=3.78, p<0.056, did not reach .05.

Authors: Shamsi T. Iqbal, Brian P. Bailey (University of Illinois)

Published in: CHI 2008

How the study was done: 2 studies of a system that holds notifications until a break in the task. 16 users, 8 per task domain.

Used in the design: Browser use in conversations, Serve Kyberna’s local models to other Macs and devices.

Read the paper

Enhanced active choice: A new method to motivate behavior change

Highlighting what the unpreferred option loses moved more people to the favored option than a plain choice; the paper proposes it where a default is not feasible. Compliance was 93% against 52% in the laboratory, F(1,58)=15.58, p<.01, partial eta squared .212 (computed); on a web page 21.9% against 12.3%.

Authors: Punam Anand Keller (Dartmouth College), Bari Harlam (CVS/Caremark), George Loewenstein (Carnegie Mellon University), Kevin G. Volpp (University of Pennsylvania)

Published in: Journal of Consumer Psychology 21, 2011

How the study was done: Studies including 2 field experiments comparing opt-in, active choice and enhanced active choice. 5,491 members got the opt-in message and 4,459 the enhanced active choice message.

Used in the design: Setup sheet, Recovery key, Local models, Encrypted Record, Move to a new Mac.

Read the paper

Evaluating the End-User Experience of Private Browsing Mode

A disclosure saying tabs vanish did not say where data is deleted, and several participants read it as removal from web servers too. The paper gives no effect size for this finding.

Authors: Ruba Abu-Salma (University College London), Benjamin Livshits (Imperial College London, Brave Software)

Published in: CHI 2020

How the study was done: A usability inspection in 3 parts with interviews and a quiz on private browsing. 25 participants.

Agrees with how it works: Private conversations. A Private conversation says what happens to it: nothing is written to disk, and closing the window discards it. The study looked at web browsers.

Read the paper

Explanation Beats Context: The Effect of Timing & Rationales on Users' Runtime Permission Decisions

Giving a reason made a grant more likely (odds ratio 2.73). Requests in context with a reason were granted 92% of the time, against 74% upfront without one.

Authors: Yusra Elbitar (CISPA Helmholtz Center for Information Security, Saarland University), Michael Schilling (CISPA), Trung Tin Nguyen (CISPA, Saarland University), Michael Backes (CISPA), Sven Bugiel (CISPA)

Published in: USENIX Security 2021

How the study was done: A comparative online study with mockup apps, varying when a permission was requested and whether a reason was given. 473 participants on Amazon Mechanical Turk, 120 variations of 30 apps.

Used in the design: Kyberna Gateway, Kyberna Guard, Approval requests, Remembered answers, Site approvals, Rogue Agent Protection, Reminder file access, Commands run by local and API models, Explain This Request, Background tasks list, Agents a conversation starts, Writing guides, Messages between conversations.

Read the paper

Exploring decision making with Android's runtime permission dialogs using in-context surveys

Participants accepted 84% of permission requests and said they were comfortable with 90% of what they granted.

Authors: Bram Bonné (Hasselt University), Sai Teja Peddinti, Igor Bilogrevic, Nina Taft (Google)

Published in: SOUPS 2017

How the study was done: A longitudinal experience-sampling survey on Android runtime permission dialogs. 157 participants over 6 weeks.

Used in the design: Messages between conversations.

Read the paper

Harder to Ignore? Revisiting Pop-Up Fatigue and Approaches to Prevent It

Across 4 levels of exposure, 2 attractors that worked in the earlier study, Reveal and AC+Delay, became less effective with repeated exposure.

Authors: Cristian Bravo-Lillo, Lorrie Cranor, Saranga Komanduri (Carnegie Mellon University), Stuart Schechter (Microsoft Research), Manya Sleeper (Carnegie Mellon University)

Published in: SOUPS 2014

How the study was done: An online experiment at 4 levels of exposure to a repeated dialog. Participants on Amazon Mechanical Turk.

Used in the design: Kyberna Gateway, Kyberna Guard, Tool policy, Approval requests, Remembered answers, Site approvals, Pause and Kill a Session, Rogue Agent Protection, Explain This Request, Folder Snapshots.

Read the paper

How Bad Can It Git? Characterizing Secret Leakage in Public GitHub Repositories

Secret leakage affected over 100,000 repositories, an estimated 89.10% of the secrets found were sensitive, and 81% were not removed.

Authors: Michael Meli (North Carolina State University), Matthew R. McNiece (North Carolina State University, Cisco Systems), Bradley Reaves (North Carolina State University)

Published in: NDSS 2019

How the study was done: A measurement study: a scan of public GitHub commits over nearly 6 months and a snapshot of 13% of open-source repositories. Billions of files on public GitHub.

Agrees with how it works: Hugging Face token, Slack channel. Kyberna keeps these tokens in its own sealed store on this Mac. They stay out of project folders, the place this study found keys leaking from. The study measured public GitHub.

Read the paper

Identifying the Risks of LM Agents with an LM-Emulated Sandbox (ToolEmu)

68.8% of the failures found would be valid real-world failures, and the safest agent showed severe failures 23.9% of the time.

Authors: Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Chris J. Maddison (University of Toronto, Vector Institute), Yann Dubois, Tatsunori Hashimoto (Stanford University)

Published in: ICLR 2024

How the study was done: An emulator that runs agents against simulated tools, with an automatic safety evaluator checked by human evaluation. 36 toolkits and 144 test cases.

Used in the design: Local models, Tool policy, Approval requests, Folder Snapshots.

Read the paper

Improving SSL Warnings: Comprehension and Adherence

Nearly 30% more users chose to remain safe after the new warning, though the warning was not well understood.

Authors: Adrienne Porter Felt, Alex Ainslie, Robert W. Reeder, Sunny Consolvo, Somas Thyagaraja, Alan Bettes, Helen Harris (Google), Jeff Grimes (University of Pennsylvania)

Published in: CHI 2015

How the study was done: Micro-surveys and a field experiment in pre-release Chrome. Chrome users on the pre-release channels.

Used in the design: Remembered answers, Rogue Agent Protection, Folder Boundary, Paused background runs, Messages between conversations, Permission Boundary.

Read the paper

IsolateGPT

Overhead stayed under 30% for three-quarters of the tested queries.

Authors: Yuhao Wu, Ning Zhang, Umar Iqbal (Washington University in St. Louis), Franziska Roesner, Tadayoshi Kohno (University of Washington)

Published in: NDSS 2025

How the study was done: An architecture that gives each app its own model instance and a mediated protocol, with measurements. Query workloads on the prototype.

Used in the design: Folder Boundary, Permission Boundary.

Read the paper

Large Language Models Can Be Easily Distracted by Irrelevant Context

Chain of thought on code-davinci-002 solved 95.0% of the 100 base problems and 72.4% of the GSM-IC problems; an instruction to ignore irrelevant information raised it to 77.8%. The paper reports no standardized effect size.

Authors: Freda Shi (Google DeepMind student researcher; Toyota Technological Institute at Chicago), Xinyun Chen, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, Denny Zhou (Google DeepMind), Kanishka Misra (Google DeepMind student researcher; Purdue University)

Published in: ICML 2023 (PMLR 202)

How the study was done: A benchmark of grade-school math problems with 1 irrelevant sentence added, run with several prompting methods. GSM-IC, built from 100 GSM8K problems; 4,000 problems sampled; code-davinci-002 and text-davinci-003.

Used in the design: Token Optimization.

Read the paper

LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models

On GSM8K the full prompt scored 78.85, LLMLingua 77.41 at 14 times compression and Selective Context 44.20 at 15 times. The paper reports no standardized effect size.

Authors: Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu (Microsoft Corporation)

Published in: EMNLP 2023

How the study was done: Coarse-to-fine prompt compression with a small model, against sentence selection, Selective Context and GPT-4 rewriting. GSM8K, BBH, ShareGPT and Arxiv-March23; GPT-3.5-Turbo and Claude-v1.3.

Used in the design: Token Optimization.

Read the paper

LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression

With 20 documents the 2,946-token prompt scored 75.7 with the answer first and 54.1 with it 10th; compressed 4 times it scored 75.0 and 71.2. The paper reports no standardized effect size.

Authors: Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu (Microsoft Corporation)

Published in: ACL 2024

How the study was done: Question-aware prompt compression with document reordering, against retrieval and compression baselines. NaturalQuestions with 20 documents, LongBench, ZeroSCROLLS, MuSiQue and LooGLE; GPT-3.5-Turbo and LongChat-13B.

Used in the design: Token Optimization.

Read the paper

Making Retrieval-Augmented Language Models Robust to Irrelevant Context

A random passage lowered in-context accuracy by more than 10 points on average; the NLI filter stopped the loss but discarded relevant passages too. The paper reports no standardized effect size.

Authors: Ori Yoran, Tomer Wolfson (Tel Aviv University; Allen Institute for AI), Ori Ram, Jonathan Berant (Tel Aviv University)

Published in: ICLR 2024

How the study was done: Few-shot question answering with no retrieval, top-1 retrieval or a random passage, and 2 mitigations: an NLI filter and fine-tuning. Llama-2-13B (and 70B on NQ) on 5 question-answering sets.

Used in the design: Token Optimization.

Read the paper

Measuring the Vague Meanings of Probability Terms

The vague meanings of probability words can be measured as functions over the 0 to 1 probability range.

Authors: Thomas S. Wallsten (University of North Carolina at Chapel Hill), David V. Budescu, Amnon Rapoport (University of Haifa), Rami Zwick, Barbara Forsyth (University of North Carolina at Chapel Hill)

Published in: Journal of Experimental Psychology: General 115(4), 1986

How the study was done: 2 experiments with a modified pair-comparison procedure on probability words. Experimental subjects judging probability terms.

Used in the design: Writing guides.

Read the paper

MELON

It gave better prevention and utility than the compared defenses. On GPT-4o, MELON-Aug kept 68.72% utility with 0.32% attack success, against 69.08% and 16.06% with no defense.

Authors: Kaijie Zhu, Xianjun Yang, Jindong Wang, Wenbo Guo, William Wang

Published in: ICML 2025

How the study was done: A defense that re-executes an agent's turn with masked input, evaluated on AgentDojo. AgentDojo's tasks.

Used in the design: Folder Boundary, Permission Boundary.

Read the paper

MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models

Average scores fell from single-turn to multi-turn for every model, GPT-3.5-Turbo from 8.07 to 7.23 and GPT-4 from 9.17 to 8.84. The paper reports no standardized effect size.

Authors: Wai-Chung Kwan, Kam-Fai Wong (The Chinese University of Hong Kong), Xingshan Zeng, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu (Huawei Noah's Ark Lab), Yuxin Jiang (The Hong Kong University of Science and Technology)

Published in: EMNLP 2024

How the study was done: A benchmark that scores each multi-turn query against its single-turn version, with distracting turns inserted. 1,170 multi-turn queries; 11 models.

Used in the design: Token Optimization.

Read the paper

OS-Harm

All tested models complied with much misuse, were vulnerable to static injection and sometimes acted unsafely unprompted. o4-mini complied with a static injection in 20% of cases; Claude 3.7 Sonnet was unsafe on 70% of misuse tasks; the average unsafe rate was 28%.

Authors: Thomas Kuntz, Agatha Duzan, Hao Zhao, Francesco Croce (EPFL), J. Zico Kolter (Carnegie Mellon University), Nicolas Flammarion, Maksym Andriushchenko (EPFL)

Published in: NeurIPS 2025 Datasets and Benchmarks

How the study was done: A benchmark of computer use agents across misuse, injection and misbehavior. 150 tasks.

Used in the design: Folder Boundary, Permission Boundary.

Read the paper

PaperTrail: A Claim-Evidence Interface for Grounding Provenance in LLM-based Scholarly Q&A

Showing which claims had support lowered trust in the answers, t(25)=2.61, p=0.015, d=0.44, with no significant change in reliance, W=146, p=0.313.

Authors: Anna Martin-Boyle (University of Minnesota), Cara A.C. Leckey, Martha C. Brown (NASA Langley Research Center), Harmanpreet Kaur (University of Minnesota)

Published in: CHI 2026

How the study was done: A within-subjects study of an interface marking supported and unsupported claims, against a baseline. 26 researchers doing 2 scholarly editing tasks.

Agrees with how it works: Look up in this folder. Look up in this folder says which claims nothing supports. The study tested the same kind of display with researchers, not with a general audience.

Read the paper

Predicting short-term stock fluctuations by using processing fluency

Students expected easy names to gain 3.90% and hard names to lose 3.86% over a year. Real shares with harder names did worse after 1 day of trading.

Authors: Adam L. Alter, Daniel M. Oppenheimer (Princeton University)

Published in: PNAS 103(24), 2006

How the study was done: A laboratory study on invented share names and 2 field studies of real share names. 29 undergraduates; 89 NYSE shares, then 665 NYSE and 116 AMEX shares.

Used in the design: Kyberna Gateway.

Read the paper

RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation

On NQ, exact match was 21.99 with no documents, 39.39 with the top 5 documents in 660 tokens and 36.57 with the extractive compressor in 37 tokens. The paper reports no standardized effect size.

Authors: Fangyuan Xu, Eunsol Choi (The University of Texas at Austin), Weijia Shi (University of Washington)

Published in: ICLR 2024

How the study was done: Compressors that extract sentences from or summarize retrieved documents, or return nothing, before the model reads them. Flan-UL2 on NQ, TriviaQA and HotpotQA, and language modeling on WikiText-103.

Used in the design: Token Optimization.

Read the paper

SecAlign

Attack success stayed mainly under 10%, including against optimization-based attacks.

Authors: Sizhe Chen (UC Berkeley, Meta), Arman Zharmagambetov (Meta), and colleagues

Published in: CCS 2025

How the study was done: A defense that fine-tunes a model by preference optimization against prompt injection. Evaluations on prompt injection benchmarks.

Used in the design: Folder Boundary, Permission Boundary.

Read the paper

See No Evil: Phishing for Permissions with False Transparency

In the survey, 41% agreed an app could prompt again for a permission it already held. 18 of the 20 laboratory subjects granted every malicious request.

Authors: Güliz Seray Tuncay (Google, University of Illinois at Urbana-Champaign), Jingyu Qian, Carl A. Gunter (University of Illinois at Urbana-Champaign)

Published in: USENIX Security 2020

How the study was done: An online survey and a laboratory study of permission requests that hide which app asks. 200 survey participants and 20 laboratory subjects.

Used in the design: Kyberna Gateway.

Read the paper

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

The interface reached pass@1 rates of 12.5% and 87.7%, and the paper reports that interface design affects agent performance.

Authors: John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press (Princeton University)

Published in: NeurIPS 2024

How the study was done: An agent system with a purpose-built interface, measured on 2 benchmarks. SWE-bench and HumanEvalFix.

Used in the design: Token Optimization.

Read the paper

The Cost of Interrupted Work: More Speed and Stress

Interrupted people finished in less time with no loss of quality, and they reported more stress, frustration, time pressure and effort. Mean task time was 22.77 minutes without interruptions, 20.31 in the same context and 20.60 in a different one; stress rose, F(2,92)=12.15, p<.001, partial eta squared .209 (computed).

Authors: Gloria Mark (University of California, Irvine), Daniela Gudith, Ulrich Klocke (Humboldt University Berlin)

Published in: CHI 2008

How the study was done: A laboratory experiment on email work with interruptions. Participants in a controlled experiment; the count was not found in the text read.

Used in the design: Browser use in conversations, Serve Kyberna’s local models to other Macs and devices.

Read the paper

The Effect of Developer-Specified Explanations for Permission Requests on Smartphone User Behavior

Permission requests with an explanation were significantly more likely to be approved, and around 19% of requests included one.

Authors: Joshua Tan (North Dakota State University), Khanh Nguyen (University of California, Riverside), Michael Theodorides, Christopher Thompson, Serge Egelman, David Wagner (University of California, Berkeley), Heidi Negrón-Arroyo (University of Puerto Rico, Mayagüez)

Published in: CHI 2014

How the study was done: An online survey of smartphone users, an analysis of iOS apps and a survey of developers. 772 smartphone users, 4,400 iOS apps and 30 developers.

Used in the design: Kyberna Gateway.

Read the paper

The Goal-Gradient Hypothesis Resurrected

People speeded up as they neared a reward: the time between first and last observed purchase fell by 0.7 days (t = 2.6, p < .05).

Authors: Ran Kivetz, Oleg Urminsky (Columbia University), Yuhuang Zheng (Fordham University)

Published in: Journal of Marketing Research 43, 2006

How the study was done: Field studies of a café reward program and other tasks. 108 café customers in 1 study.

Used in the design: Setup sheet, Recovery key, Local models, Encrypted Record.

Read the paper

The Importance of Percent-Done Progress Indicators for Computer-Human Interfaces

People preferred having a progress indicator, a highly significant difference. 86.1% of subjects liked progress indicators; the difference was significant, pr=0.0006.

Authors: Brad A. Myers (University of Toronto)

Published in: CHI 1985

How the study was done: A formal experiment on database queries with and without a percent-done progress indicator. 48 subjects, most of them computer science graduate students, in groups each using 2 versions.

Used in the design: Recovery key, Model servers, Rogue Agent Protection, Turn status line, Send checks, Move to a new Mac, Agents a conversation starts, Serve Kyberna’s local models to other Macs and devices, Messages between conversations.

Read the paper

The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination

Gains in task performance came with higher rates of invented tool use, and reducing the invented use cost utility. Tool hallucination rose from 34.8% to 90.2% and from 54.7% to 100.0% under 2 reasoning methods; on Qwen3 4B, from 3.4% to 29.4%.

Authors: Chenlong Yin (The Pennsylvania State University), Zeyang Sha (Nanjing University of Science and Technology), Shiwen Cui, Changhua Meng (independent researchers), Zechao Li (Nanjing University of Science and Technology)

Published in: ACL 2026

How the study was done: Controlled experiments with a benchmark of tool hallucination across reasoning methods. RL, distillation and toggleable reasoning modes.

Used in the design: What each engine keeps, Local models, Encrypted Record, Channel modes, Prompt Injection Prevention, Meeting notes, Meeting action items, Privacy Protection.

Read the paper

Using implementation intentions prompts to enhance influenza vaccination rates

Of those with no prompt, 33.1% were vaccinated. A prompt for date and time raised the rate by 4.2 percentage points (P = 0.03); a date alone raised it by 1.5, not significant (P = 0.51).

Authors: Katherine L. Milkman (University of Pennsylvania), John Beshears (Stanford University), James J. Choi (Yale University), David Laibson, Brigitte C. Madrian (Harvard University)

Published in: PNAS 108(26), 2011

How the study was done: A field experiment on mailers about flu vaccination clinics. 3,272 employees of a large Midwestern utility.

Used in the design: Meeting action items.

Read the paper

What the App is That?

On stock Android, 19.19% and 17.17% detected a spoofed app. With a system-drawn identity indicator, 64.52% and 76.34% detected it.

Authors: Antonio Bianchi, Jacopo Corbetta, Luca Invernizzi, Yanick Fratantonio, Christopher Kruegel, Giovanni Vigna (University of California, Santa Barbara)

Published in: IEEE S&P 2015

How the study was done: An analysis of Android's interface attacks and an online experiment on an emulated phone. 308 participants.

Used in the design: Kyberna Gateway.

Read the paper

When It's Better to Ask Forgiveness than Get Permission: Attribution Mechanisms for Smartphone Resources

31% (12 of 39) named the responsible app without attribution, against 81% (29 of 36) with a notification naming it.

Authors: Christopher Thompson, Maritza Johnson, Serge Egelman, David Wagner, Jennifer King (University of California, Berkeley)

Published in: SOUPS 2013

How the study was done: An online survey and a controlled laboratory experiment that simulated app misbehavior. 76 people in the laboratory experiment.

Used in the design: Kyberna Gateway, Kyberna Guard, Approval requests, Remembered answers, Site approvals, Rogue Agent Protection, Explain This Request.

Read the paper

Why Do Multi-Agent LLM Systems Fail?

Agreement reached 0.88, and failures fell into 14 modes in 3 categories: system design, inter-agent misalignment and task verification.

Authors: Mert Cemri, Melissa Z. Pan, Lakshya A Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, Ion Stoica (UC Berkeley), Shuyi Yang (Intesa Sanpaolo)

Published in: NeurIPS 2025 Datasets and Benchmarks

How the study was done: An annotated study of multi-agent system traces with a taxonomy validated by experts. 150 traces analyzed by hand and over 1,600 annotated across 7 frameworks.

Used in the design: Local models.

Read the paper

Why Phishing Works

23% of participants did not look at browser-based cues, leading to incorrect choices 40% of the time.

Authors: Rachna Dhamija (Harvard University), J. D. Tygar, Marti Hearst (University of California, Berkeley)

Published in: CHI 2006

How the study was done: A usability study: participants judged which web sites were fraudulent. 22 participants, 20 web sites.

Used in the design: Kyberna Gateway.

Read the paper

You've Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings

79% heeded active warnings, while 1 participant heeded the passive warning; 97% fell for at least 1 phishing message.

Authors: Serge Egelman, Lorrie Faith Cranor, Jason Hong (Carnegie Mellon University)

Published in: CHI 2008

How the study was done: A laboratory study with a simulated spear phishing attack. 60 participants.

Used in the design: Kyberna Gateway, Projects, Folder switching, What each engine keeps, APIs, Model servers, MCP servers, Loaded Models, Memory check, Approval requests, Remembered answers, MCP server checks, Rogue Agent Protection, Folder Boundary, Paused background runs, Commands run by local and API models, Background tasks list, Agents a conversation starts, Messages between conversations, Permission Boundary, Safe Rewrite, Privacy Protection.

Read the paper

Your Attention Please

Inhibitive attractors, which hold a person on the important text, made people significantly less likely to ignore harmful clues, reducing the less safe choice by up to 50%.

Authors: Cristian Bravo-Lillo, Lorrie Faith Cranor, Julie Downs, Saranga Komanduri, Robert W. Reeder, Manya Sleeper (Carnegie Mellon University), Stuart Schechter (Microsoft Research)

Published in: SOUPS 2013

How the study was done: 3 between-subjects experiments with installation dialogs that had attractors added. Participants on Amazon Mechanical Turk; experiment 3 recruited 878.

Used in the design: Kyberna Gateway, What each engine keeps, Model servers, Kyberna Guard, Approval requests, Remembered answers, Site approvals, Rogue Agent Protection, Folder Boundary, Paused background runs, Explain This Request, Messages between conversations, Permission Boundary, Safe Rewrite, Ask About This Request, Privacy Protection.

Read the paper

Your Reputation Precedes You

Users heeded warnings about sites they had not visited; on some days they ignored more than half of warnings about sites they had visited. Click-through was 25.4% on previously visited sites against 13.1% on new sites (p<0.0001).

Authors: Hazim Almuhimedi (Carnegie Mellon University), Adrienne Porter Felt, Robert W. Reeder, Sunny Consolvo (Google)

Published in: SOUPS 2014

How the study was done: Field data on Chrome's malware warning and an online survey-based experiment. Chrome field data and an online survey.

Used in the design: Kyberna Gateway, Projects, Folder switching, Rogue Agent Protection, Folder Snapshots.

Read the paper

← Back to Features