{"id":"benchmark-picker","repo":"AStox/paseo-model-bench","url":"https://github.com/AStox/paseo-model-bench","package":"paseo-benchmark-picker","npm":{"package":"paseo-benchmark-picker","version":"0.1.2","integrity":"sha512-1BOwkfC0VKz6skS+X2ZekLbfAIK0VjS/Rz1h1XylvtCjQ+bBbErCqQqfGy+b79+9gZRtwRvTVdPQhPEQ/UZ6KQ==","publishedAt":"2026-09-29T17:46:19.807Z","downloadsLast30Days":0},"name":"benchmark-picker","description":"Pick coding models by benchmark score and cost in Paseo.","categories":["productivity"],"platforms":[],"caveats":["If your daemon has a password, set PASEO_PASSWORD on the daemon so the plugin can switch models.","The desktop picker takeover relies on Paseo test IDs and falls back to the normal dropdown if they change."],"images":["https://raw.githubusercontent.com/AStox/paseo-model-bench/1d5d1060f8177beec3f4aa54ff97e4b2b4748851/docs/screenshot.png"],"themes":[],"health":{"manifestValid":true,"hasReadme":true,"hasLicense":false,"hasTests":false,"hasTypecheckScript":true,"updatedRecently":true},"scannedAt":"2026-09-29T19:51:48.642Z","addedAt":"2026-09-29T22:50:27+03:00","npmSecurity":{"status":"passed","blockingFindings":0,"advisoryFindings":0,"scannedAt":"2026-09-29T19:51:38.289Z","version":"0.1.2","integrity":"sha512-1BOwkfC0VKz6skS+X2ZekLbfAIK0VjS/Rz1h1XylvtCjQ+bBbErCqQqfGy+b79+9gZRtwRvTVdPQhPEQ/UZ6KQ=="},"version":"0.1.2","security":{"status":"passed","blockingFindings":0,"advisoryFindings":0,"scannedAt":"2026-09-29T19:51:38.289Z","commit":"1d5d1060f8177beec3f4aa54ff97e4b2b4748851"},"paseoVersionRequirement":">=0.8.0","descriptionNodes":[{"type":"text","text":"Pick coding models by benchmark score and cost in Paseo."}],"caveatNodes":[[{"type":"text","text":"If your daemon has a password, set PASEO_PASSWORD on the daemon so the plugin can switch models."}],[{"type":"text","text":"The desktop picker takeover relies on Paseo test IDs and falls back to the normal dropdown if they change."}]],"manifest":{"id":"benchmark-picker","requirements":{"paseo":">=0.8.0"},"build":[["npm","install","--omit=dev"]]},"repoMeta":{"stars":4,"defaultBranch":"main","pushedAt":"2026-09-29T17:43:30Z"},"owner":{"login":"AStox","avatarUrl":"https://avatars.githubusercontent.com/u/6720265?v=4"},"installNotesHtml":"

Needs Paseo 0.8 or later with plugins turned on (Settings -> Plugins).

\n

On Paseo 0.9 or later:

\n
paseo plugin add npm:paseo-benchmark-picker\n
\n

On Paseo 0.8:

\n
paseo plugin add AStox/paseo-model-bench\n
\n

From a checkout:

\n
npm install\nnpm run typecheck\npaseo plugin install /absolute/path/to/paseo-model-bench\n
\n

After editing source, paseo plugin reload benchmark-picker.

","readmeText":"# Benchmark Picker\n\nPaseo plugin. It turns the model picker into a score vs cost chart, so you can see what you're actually paying for before you switch models.\n\n![Benchmark Picker popover](docs/screenshot.png)\n\n\n\nhttps://github.com/user-attachments/assets/2c956303-d770-417b-852b-31478a208495\n\n\n\nEach line is one model from your chat's provider. Each point on it is a reasoning effort (low, medium, high, etc). Hover a point to see its score and cost, and click it to switch the chat to that model and effort (on mobile, tap once to preview and again to switch). Hover a model chip to pick its line out of the crowd.\n\n## benchmarks\n\nPick one from the title dropdown. Your pick sticks.\n\n- Terminal-Bench 4.0 (default)\n- AA Intelligence Index\n- DeepSWE v1.1\n- ARC-AGI-2\n- CursorBench\n- FrontierSWE\n- WeirdML\n- ProofBench\n\nTerminal-Bench 4.0 and the AA Intelligence Index come from [Artificial Analysis](https://artificialanalysis.ai/models). They run every reasoning effort for most big models, which is why Terminal-Bench is the default. They dont have a public API without a key, so the plugin reads the data embedded in one of their model pages. If they redesign that page, those two go empty and the rest keep working. ARC-AGI-2 comes from ARC Prize's public leaderboard files. Everything else comes from [Epoch AI's benchmark data](https://epoch.ai/benchmarks) (CC BY 4.0). No API keys needed. The plugin downloads all of it on the daemon and refreshes them every hour.\n\nNot every model gets run on every benchmark, and results without a published cost get dropped. So if a model you care about is missing, thats usually why.\n\n## how it works\n\n- On desktop and web, clicking the model name or the thinking level in the composer opens the chart right where the dropdown would've been. Shift-click still gets you the normal dropdown.\n- On mobile it's a Bench pill above the composer instead. Plugins cant replace built-in controls, so the desktop takeover is a click intercept on Paseo's own test IDs. If Paseo renames those, you just get the normal dropdowns back.\n- Models without results on the current benchmark are still one click away. Hit the list icon in the header (or \"All N models\" under the chart). It works like the normal picker: pick a model, then an effort.\n- The gear in the header opens the same provider settings window as the gear in Paseo's picker (discovered models, add model, diagnostic, refresh).\n- New sessions work too. A draft has no agent yet, so there the plugin shows its own popup with a provider switcher, and when you pick something it clicks through Paseo's picker for you.\n- Click a model chip in the legend to hide it. The chart rescales to whatever is left, and hidden models are remembered on the daemon.\n- Switching models goes through the daemon's websocket, same as the CLI, since the plugin SDK doesnt have a model setter yet. If your daemon has a password, set `PASEO_PASSWORD` on the daemon.\n\n## Install\n\nNeeds Paseo 0.8 or later with plugins turned on (Settings -> Plugins).\n\nOn Paseo 0.9 or later:\n\n```bash\npaseo plugin add npm:paseo-benchmark-picker\n```\n\nOn Paseo 0.8:\n\n```bash\npaseo plugin add AStox/paseo-model-bench\n```\n\nFrom a checkout:\n\n```bash\nnpm install\nnpm run typecheck\npaseo plugin install /absolute/path/to/paseo-model-bench\n```\n\nAfter editing source, `paseo plugin reload benchmark-picker`.\n"}