Malva

Instead, tools capture screenshots or UI trees, translate model outputs into GUI actions, and report whether the environment changed as expected.</p><p dir="ltr">The software loop contains three main parts:</p><ol><li><b>Agent.</b> The LLM, planner, memory, observations, action history, reasoning outputs, and selected tool calls form the agent-side state used to plan the next step.</li><li><b>Environment.</b> The live web, desktop, or mobile GUI includes widgets, windows, focus, permissions, loading state, network conditions, and received tool-side effects.</li><li><b>Tools.</b> Tools mediate between the agent and the environment.

They observe GUI state, ground targets, parse and validate model outputs, cache intermediate state, invoke backend adapters such as clicking, typing, or scrolling, and return observations and feedback to the agent.</li></ol><h3 dir="ltr">Artifact Contents</h3><h4 dir="ltr">Application Benchmark</h4><p dir="ltr">`application.csv` contains the 40 open-source agentic GUI applications studied in the paper. The applications span web, desktop, and mobile environments.</p><p dir="ltr">The file records:</p><ol><li>software project name</li><li>GitHub link and commit ID</li><li>application classification</li><li>deployment environment and application scope</li><li>used LLM and observation type</li></ol><h4 dir="ltr">Task Suite</h4><p dir="ltr">`task.csv` contains 62 executable tasks translated from issue scenarios.

Each task is tied to one application and specifies an initial state and a verifiable success criterion. The tasks cover 14 environment-specific scenarios.</p><p dir="ltr">The file records:</p><ol><li><b>Prompt:</b> the natural-language instruction given to the agent</li><li><b>Type:</b> the agent category, such as web, desktop, or mobile agent</li><li><b>Application:</b> the target application for the task</li><li><b>Initial State</b><b>:</b> the required starting condition before execution</li><li><b>Success Criteria:</b> the condition used to judge task completion</li><li><b>Number of Steps:</b> whether the task is single-step or multi-step</li><li><b>Scenario:</b> the environment-specific scenario the task belongs to</li></ol><p><br></p><h4 dir="ltr">Malva Defect Library</h4><p dir="ltr">`defect.csv` contains the 335 confirmed interaction failures used in the paper.

The failures come from 40 applications and are organized by defect pattern, root cause, consequence, source-code location, and defect-triggering task.</p><p dir="ltr">For defect types that appear in several forms, the file separates cases with a blank line and labels them as case 1, case 2, and so on.</p><p dir="ltr">The columns in `defect.csv` are:</p><ol><li><b>APP: </b>the application from GitHub</li><li><b>commit url:</b> the relevant version of the application</li><li><b>issues:</b> the corresponding GitHub issue link, if available</li><li><b>status:</b> the issue status, such as open or closed</li><li><b>types:</b> the defect type</li><li><b>cases:</b> different cases for the same defect type</li><li><b>explanation:</b> the defect explanation</li><li><b>consequences:</b> the failure impact</li><li><b>source-code locations:</b> the relevant source-code location</li><li><b>defect-triggering tests:</b> the input task that triggers the defect</li><li><b>rootcause:</b> the underlying root cause The consequence abbreviations are: ST for fail-stop, IC for incorrectness, SL for slower execution, UI for user-interface disruption, TK for increased token usage, and IS for security exposure.</li></ol><h4 dir="ltr">MalvaGuard</h4><p dir="ltr">`malvaguard/` contains MalvaGuard, a pre-deployment static checker derived from the Malva taxonomy.

MalvaGuard looks for code-level anti-patterns where tools observe, invoke, retry, or accept GUI interactions without guards that keep the agent state aligned with the live environment.</p><p dir="ltr">MalvaGuard has two stages:</p><ol><li><b>Contract checking.</b> It locates source-code or prompt regions that contain tool-environment interactions, then applies rule-based checks for required guards, including observation bounds, progress checks, action validation, and feedback verification.</li><li><b>Defect reporting.</b> It groups related missing-contract warnings into bug-level defect reports.

Each report records where the potential defect appears, what guard is missing, which taxonomy pattern it maps to, what warning locations support the report, and what guard pattern developers can add.</li></ol><p dir="ltr">The workbook `malvaguard/malvaguard_results.xlsx` contains the current MalvaGuard results used in the paper. On an additional 20-application corpus with no overlap with the 40 applications used to build the defect library, MalvaGuard flags 497 missing-contract locations and groups them into 150 bug-level defect candidates.

Of these candidates, 86 match oracle defects and 64 do not, yielding 57.3% precision. The oracle contains 112 real defects, and MalvaGuard reports 86 while missing 26, yielding 76.8% recall.</p><h3 dir="ltr">Running MalvaGuard</h3><p dir="ltr">From the repository root:</p><p dir="ltr">```bash</p><p dir="ltr">python -m malvaguard --list-rules</p><p dir="ltr">python -m malvaguard path/to/agentic-gui-project</p><p dir="ltr">python -m malvaguard path/to/agentic-gui-project --format json</p><p dir="ltr">python -m malvaguard path/to/agentic-gui-project --level findings</p><p>```</p><p dir="ltr">By default, MalvaGuard reports grouped defect candidates.

Use `--level findings` to inspect raw missing-contract warning locations.</p><h3 dir="ltr">Quick Start: Using the Malva Defect Library</h3><p dir="ltr">In our paper, we introduce UI-TARS-desktop, a vision-language desktop agent. When sending a WhatsApp message, the agent keeps its own transparent overlay window on top of the screen. The text entry action succeeds because the text field remains unobstructed.

The next click, however, is routed to the agent overlay instead of the WhatsApp send button. The agent then believes the message was sent, while the live GUI never delivered it.</p><p dir="ltr">To inspect this example:</p><ol><li>Open `application.csv` to find the GitHub link and commit ID for UI-TARS-desktop.</li><li>Open `defect.csv` to inspect the defect pattern, source-code location, impact, and linked issue if available.</li><li>Open `task.csv` to find the executable task, initial state, and success criteria used to reproduce the failure.</li><li>Run or review the task to understand how the tool-mediated mismatch occurs.</li></ol><p dir="ltr"><br></p>