Repository navigation
Conversation
Recipes and other step-by-step answers now render as an interactive card: servings that rescale amounts with friendly fractions and whole eggs, US/metric units, timer chips on every duration, copy/share/print, and a full-screen guided view with a big countdown.
… cards
A recipe or workout answer from a small local model was a long wall of
list text. The flagship now has four tools (show_recipe, show_workout,
show_how_to, show_checklist) that send the answer as a `guide` render
card, so the reply only adds a tip or two. A bad argument comes back as
{"ok": false, "error": ...} naming the fix, so the model can call again.
The flagship sidecar's /query stream never set tool_result.render, so no
card tool could ever draw over HTTP. Its translator now gets the render
map.
- GuideToolsMixin, registered in KNOWN_TOOLS as `guide`, wired into
GaiaAgent and a `guide` bundle in the full profile's tool loader.
- tools_count 95 -> 99; capability matrix regenerated.
- `guide` row in the SSE contract §4.3; docs and changelogs updated.
Cards drew only through the sidecar relay, vanished on reload, and a small model's real output broke scaling and temperatures. - The SSE handler now names the card when it lifts one, so in-process turns draw it and a failed call never draws an "Invalid payload" card. The translator reads that instead of its own tool map. - A reloaded chat rebuilds its cards from the saved tool steps, and the post-turn refetch keeps them instead of wiping them. - Ingredients written "Olive oil (1 tablespoon)" now scale. - "165°F/74°C" pairs no longer convert into duplicates, and doneness temperatures stay exact instead of snapping to 5 degrees. - The tool result tells the model the card is on screen, so it stops re-typing the recipe under it.
…de cards Gemma wrote "Step 1: Turn off the water", so the card read "1 Step 1: ...".
Trip plans now come back as a day-by-day card, and tip, bill-split and loan
questions open a calculator card that does the math, so a small model's
arithmetic never reaches the answer. The model only sends starting numbers.
- show_itinerary: a fifth guide type (days as steps, packing as items).
- CalculatorToolsMixin (KNOWN_TOOLS["calculator"]): show_tip_calculator and
show_loan_calculator validate ranges and return a `calculator` render card;
a bad number comes back as {"ok": false, "error": ...} naming the fix.
- Wired into GaiaAgent; `calculator` bundle; tools_count 99 -> 102; capability
matrix regenerated; `calculator` row in the SSE contract §4.3.
- show_recipe's ingredients description trimmed back under the per-call
token budget, which it had gone over.
- Itineraries draw as a timeline: one dot per day, titled by its "Day 1 - Lisbon" lead, with a checkable packing list. splitTitle accepts "Day N" leads only for this type; other types still treat a lead ending in a digit as body text. - The calculator card computes tip, total and per-person share, or the monthly payment (standard amortization, 0% handled), total paid and interest. Number boxes and sliders recompute live, money is formatted in the person's locale and currency, and copy uses the guide's plain-text pattern. A bad payload shows the unsupported card.
Live with Gemma, a mortgage on a 300,000 EUR flat with 60,000 down came out for a 180,000 loan: the model subtracted the down payment from `amount` and the card subtracted it again. A dollar bill split right after showed euros, since the card guessed currency from the browser region and the model then carried EUR over from the earlier turn. - `show_loan_calculator` takes `price` and says never to subtract the down payment. - Both calculators take `currency` (ISO code, "$" is USD) and the card formats in it instead of guessing from the locale.
…ide cards From a review of the branch: - A finished turn kept only the cards rebuilt from saved steps, so a live confirmation or question card beside a recipe card vanished. The live list wins again; saved steps still restore cards on reload. - Loan terms start at 1 year on both sides; a fractional term made a stuck slider and, under a month, a zero payment. - Saved checkmarks that fail to read or write now warn instead of passing silently. - The Agent UI guide says what each card does; a plan doc no longer names the removed translator map.
… can do Nothing in the app said recipes, workouts or calculators come back as cards. - A "Try" row on the new chat screen fills the box with one example per card type, to edit or send. - The first recipe, workout or how-to card carries a short tip naming its hidden controls. Dismissing it, or using one of those controls, hides it on every card for good.
…ecipe-artifact-ebf171 # Conflicts: # hub/agents/chat/python/CHANGELOG.md # src/gaia/apps/webui/src/components/ChatView.tsx # src/gaia/apps/webui/src/components/NewChat.tsx
A relayed turn showed no label for the seven new card tools, which the sidecar contract test requires for every registered tool.
The chip renders "Memory off" one commit before its effect clears the draft, so a loaded CI runner read the flag in between and failed.
|
Verdict: Approve with suggestions Recipes, workouts, how-tos, checklists, itineraries and the tip and loan calculators now show up as interactive cards. The calculators do the math in the app, so the small model's arithmetic no longer reaches the answer. Each tool checks its arguments and returns an error that says how to fix the call. Cards rebuild from the saved steps after a reload, and a failed call no longer draws an "Invalid payload" card. Both have tests. The PR includes before/after screenshots and names the eval categories to watch on the next RC. 🟡 One changelog line is now wrong for integrators. It says the stdio transport "still never sends" the card type. This PR makes stdio send it too. Stdio uses the same output handler and translator as HTTP, and the handler now attaches the card type itself. Please correct that sentence so integrators on stdio know to expect 🔍 Technical details
|
The changelog and the contract said stdio never sends render; the handler now names the card itself, so stdio carries it as HTTP does.
Main added three audio-generation tools while this branch added seven card tools; the counts become 105 tools in 28 bundles, and the gaia guide now counts the audio tools it had missed.
The manifest, registration and capability-matrix test all agree on 105 (98 plus five guide and two calculator tools); the changelog said 102.
|
Thanks @tylerbea — this is a lovely piece of work. Moving the arithmetic out of the model and into the card is exactly the right call for a small local model, and the live Gemma-4-E4B pass clearly paid off (the "Olive oil (1 tablespoon)" and double-subtracted down payment fixes are the kind of thing only a real run finds). One gap before this can merge: outside the Agent UI, a recipe or calculator answer now loses its content. The tools tell the model "the card is on the user's screen, don't repeat the list / don't state the results", but the TUI has no Either of these would close it: a TUI renderer for the two keys (a readable list for I merged 🔍 Technical details
|
…tifact-ebf171 # Conflicts: # hub/agents/gaia/npm/CHANGELOG.md
Main added three email tools and an email_threads bundle while this branch added seven card tools; the counts become 108 tools in 29 bundles.
…tifact-ebf171 # Conflicts: # hub/agents/chat/python/CHANGELOG.md
…tifact-ebf171 # Conflicts: # hub/agents/chat/python/CHANGELOG.md
…tifact-ebf171 # Conflicts: # hub/agents/chat/python/CHANGELOG.md
…tifact-ebf171 # Conflicts: # hub/agents/gaia/npm/CHANGELOG.md
Main moved the tool bundles into gaia_agent; the guide and calculator bundles move with them, and the chat package's changelog no longer claims them.
Main started attaching diff cards in the handler too, so the handler names every card and the translator's tool-map fallback stays gone. The guide counts 108 tools on a larger model; Gemma keeps the card tools.
…tifact-ebf171 # Conflicts: # docs/guides/agent-ui.mdx # docs/spec/agent-ui-query-sse-contract.md # src/gaia/apps/webui/src/components/ChatView.tsx # src/gaia/apps/webui/src/utils/chatStream.ts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tifact-ebf171 # Conflicts: # src/gaia/apps/webui/src/components/__tests__/PrivateChip.test.tsx
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's new check fails any screen that writes English straight into JSX. The recipe, workout and calculator card buttons, timers and guided-view labels now go through useT, with Arabic text for each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tifact-ebf171 # Conflicts: # docs/guides/gaia.mdx # hub/agents/gaia/CAPABILITY_MATRIX.md # hub/agents/gaia/npm/SKILL.md # hub/agents/gaia/python/gaia-agent.yaml # hub/agents/gaia/python/gaia_agent/__init__.py # hub/agents/gaia/python/tests/test_capability_matrix.py
…yout
A recipe or loan answer in the TUI showed a truncated JSON dump, because
the tools tell the model the card carries the content. The TUI now draws
every line of a guide and computes a calculator's results with the same
formulas as the Agent UI; a bad payload shows the invalid-card fallback.
In an Arabic chat, English ingredient and step lines rendered reordered
("cups flour 2") with bullets on the wrong side. Each line now orders its
own text and lines up with the card's side, and an RTL visual test covers
both cards in Arabic and English.
|
Thanks @tylerbea, this is ready for a final look. I merged One thing left for a follow-up: in Arabic, the card's labels (Ingredients, Start cooking, servings, the calculator's field and result names, the first-card tip) still show in English. 🔍 Technical details
|
Asking GAIA for a recipe, a workout, a repair guide, a packing list, a trip plan, a tip split or a loan payment used to return a wall of text, with numbers the small local model often got wrong. Now the answer arrives as an interactive card in the Agent UI. Recipes rescale with servings, switch US/metric and run step timers in a full-screen Start cooking mode. Calculators compute in the app and update as you edit. A Try row on the new chat screen and a one-time tip on the first card show people these exist. Closes #6085.
Every card type was exercised live against Gemma-4-E4B. That run caught and fixed amounts the model writes last ("Olive oil (1 tablespoon)"), "165°F/74°C" pairs, its own step numbering, re-typed recipes under the card, a double-subtracted down payment and currency carried over from an earlier turn.
gaia_tool_selection,tool_selection,gaia_core(itinerary and workout turns) andgaia_voice(the packing-list scenario expects a bulleted reply) on the next RC eval. No local run was possible here.Before and after
Start cooking (full screen, one step at a time) · Loan calculator · Try row on a phone
Test plan
npx vitest runinsrc/gaia/apps/webui: 2099 passed.npx tsc --noEmit -p .is clean.python -m pytest tests/unit/test_guide_tools.py tests/unit/test_calculator_tools.py tests/unit/test_sse_narration.py hub/agents/gaia/python/tests/test_capability_matrix.py: 187 passed.python util/lint.py --black --isort --flake8 --tool-descriptions --agentspasses.