#21 - 画面共有 — a grant, not a panel - aiko/denpa - Aiko.Works Forgejo: It is better to cultivate a Land with two Bulls, rather working under Boss who never gives Wage when asked

aiko commented

2026-07-29 17:32:08 +00:00

Owner

What to build

Card 5d. Hermes has its own vision tools, so the client only decides whether they are open. Camera and the browser panel are already gone; screen sharing survives, reduced to a single grant row.

Off — and the copy states that off means off: the client holds no capture stream, so there is nothing for her to ask for.
On — an acid indicator that never hides, showing which display, with an inline やめる; a display/window picker; and a screen.read relay row. Every frame she takes lands in the log with a thumbnail, so "when did she look" is answerable after the fact rather than a matter of trust.

Acceptance criteria

Grant off means no capture stream is held by the client
The on-indicator cannot be hidden or dismissed while sharing
Each captured frame appears in the log with a thumbnail and timestamp
やめる revokes immediately from the indicator
Tests cover that no frame can be captured while the grant is off

Blocked by

#9
#14

## What to build Card 5d. Hermes has its own vision tools, so **the client only decides whether they are open**. Camera and the browser panel are already gone; screen sharing survives, reduced to a single grant row. - **Off** — and the copy states that off means off: the client holds no capture stream, so there is nothing for her to ask for. - **On** — an **acid indicator that never hides**, showing which display, with an inline やめる; a display/window picker; and a `screen.read` relay row. **Every frame she takes lands in the log with a thumbnail**, so "when did she look" is answerable after the fact rather than a matter of trust. ## Acceptance criteria - [ ] Grant off means no capture stream is held by the client - [ ] The on-indicator cannot be hidden or dismissed while sharing - [ ] Each captured frame appears in the log with a thumbnail and timestamp - [ ] やめる revokes immediately from the indicator - [ ] Tests cover that no frame can be captured while the grant is off ## Blocked by - #9 - #14

aiko added the

ready-for-agent

label

2026-07-29 17:32:08 +00:00

aiko commented

2026-07-29 18:11:15 +00:00

Author

Owner

Reality audit

Governing rule: build to reality, Hermes leads. Not doing something is acceptable if it does not break core functionality. Breaking the design is acceptable. Building UI that does nothing is not.

Audited against the actual wire surface:

Hermes (hermes-agent 0.19.0, verified running): /health, /v1/models, /v1/runs, /v1/runs/{id}, /v1/runs/{id}/events, /v1/runs/{id}/stop, /v1/runs/{id}/approval
Run events: message.delta, tool.started, tool.completed, run.completed, run.cancelled, run.failed/run.errored, reasoning.available
Go-between → client: full-text, audio, control, error, set-model-and-conf, config-switched, config-files, background-files, history-list, history-data, new-history-created, history-deleted, user-input-transcription, backend-synth-complete, force-new-message, tool_call_status, interrupt-signal, heartbeat-ack, group-update
Client → go-between: text-input, mic-audio-data, mic-audio-end, raw-audio-data, ai-speak-signal, interrupt-signal, fetch-history-list, fetch-and-set-history, create-new-history, delete-history, fetch-configs, switch-config, fetch-backgrounds, audio-play-start, request-init-config, heartbeat

Cut: images cannot reach Hermes

HermesAgent.chat() builds its prompt with _to_text_prompt(input_data), which iterates input_data.texts only. input_data.images is never read. The Hermes path is text-only end to end.

So a captured frame has nowhere to go. There is also no message type by which Hermes could request one.

The whole grant would be an indicator, a picker and a toggle wired to nothing. Cut, and the camera and browser panel are already being removed in #9 — this makes that removal total rather than partial.

Not a loss of core functionality: she is a voice companion with tools, and her own vision tools run server-side where the client has no part to play.

Reopen if the Hermes agent path grows image support.

## Reality audit Governing rule: **build to reality, Hermes leads. Not doing something is acceptable if it does not break core functionality. Breaking the design is acceptable. Building UI that does nothing is not.** Audited against the actual wire surface: - **Hermes** (`hermes-agent 0.19.0`, verified running): `/health`, `/v1/models`, `/v1/runs`, `/v1/runs/{id}`, `/v1/runs/{id}/events`, `/v1/runs/{id}/stop`, `/v1/runs/{id}/approval` - **Run events**: `message.delta`, `tool.started`, `tool.completed`, `run.completed`, `run.cancelled`, `run.failed`/`run.errored`, `reasoning.available` - **Go-between → client**: `full-text`, `audio`, `control`, `error`, `set-model-and-conf`, `config-switched`, `config-files`, `background-files`, `history-list`, `history-data`, `new-history-created`, `history-deleted`, `user-input-transcription`, `backend-synth-complete`, `force-new-message`, `tool_call_status`, `interrupt-signal`, `heartbeat-ack`, `group-update` - **Client → go-between**: `text-input`, `mic-audio-data`, `mic-audio-end`, `raw-audio-data`, `ai-speak-signal`, `interrupt-signal`, `fetch-history-list`, `fetch-and-set-history`, `create-new-history`, `delete-history`, `fetch-configs`, `switch-config`, `fetch-backgrounds`, `audio-play-start`, `request-init-config`, `heartbeat` ### Cut: images cannot reach Hermes `HermesAgent.chat()` builds its prompt with `_to_text_prompt(input_data)`, which iterates `input_data.texts` only. **`input_data.images` is never read.** The Hermes path is text-only end to end. So a captured frame has nowhere to go. There is also no message type by which Hermes could request one. The whole grant would be an indicator, a picker and a toggle wired to nothing. **Cut**, and the camera and browser panel are already being removed in #9 — this makes that removal total rather than partial. Not a loss of core functionality: she is a voice companion with tools, and her own vision tools run server-side where the client has no part to play. Reopen if the Hermes agent path grows image support.

aiko closed this issue

2026-07-29 18:11:15 +00:00

aiko referenced this issue

2026-07-29 23:03:19 +00:00

Cut group sessions, camera, and the browser panel #9

aiko referenced this issue

2026-07-29 23:05:32 +00:00

Cut group sessions, camera, and the browser panel #9

aiko commented

2026-07-29 23:13:35 +00:00

Author

Owner

Reopening — the blocker is real, but it was named wrong

This was cut with the reopen condition "if the Hermes agent path grows image support." Hermes already has image support: _normalize_multimodal_content accepts image_url and input_image in native OpenAI vision format (gateway/platforms/api_server.py:471-547).

It is just not on the endpoint the bridge uses. Precisely:

/v1/chat/completions (_handle_chat_completions, api_server.py:3747) normalizes multimodal content. Images work.
/v1/runs (_handle_runs, api_server.py:6084+) reads raw_input[-1].get("content","") and never normalizes. conversation_history entries are coerced with str(entry["content"]). Images cannot get in.

So the client-side observation in the cut comment stands — HermesAgent.chat() builds its prompt from input_data.texts and a captured frame has nowhere to go — but the fix is not "wait for Hermes to grow vision". It is a decision about which surface the bridge talks to, and that decision is not free.

Why this is not simply unblocked

The same surface split runs the other way for approvals:

Capability	`/v1/runs` (bridge is here)	`/v1/chat/completions`
Image input	no	yes
Tool `args`	no	yes
Stable `toolCallId`	no — the bridge FIFO-pairs by tool name	yes
`approval.request` + `waiting_for_approval`	yes	no — no approval handling in that path at all
Tool result payload	no	no — emitted nowhere

_approval_notify lives inside _handle_runs (api_server.py:6250). Migrating wholesale would trade approvals — already unblocked, already scoped at #19 — for images and args. That is not obviously a win.

Actual blocked-by

One of:

/v1/runs accepts multimodal content — an upstream change to NousResearch/hermes-agent, which is the honest version of the original reopen condition.
A decided surface strategy — chat-completions for image turns and runs for the rest, or a full migration that re-solves approvals. Real architecture work, tracked at ollvt-hermes-bridge#19.

Until one lands there is still no path for a captured frame, so nothing in this issue should be built yet. Reopened because the blocker is now correctly identified and partly within our control, not because it is ready.

Consequence for #9: screen-capture code stays parked rather than being deleted. That instruction is reverted there.

## Reopening — the blocker is real, but it was named wrong This was cut with the reopen condition *"if the Hermes agent path grows image support."* Hermes already has image support: `_normalize_multimodal_content` accepts `image_url` and `input_image` in native OpenAI vision format (`gateway/platforms/api_server.py:471-547`). It is just not on the endpoint the bridge uses. Precisely: - **`/v1/chat/completions`** (`_handle_chat_completions`, `api_server.py:3747`) normalizes multimodal content. Images work. - **`/v1/runs`** (`_handle_runs`, `api_server.py:6084+`) reads `raw_input[-1].get("content","")` and never normalizes. `conversation_history` entries are coerced with `str(entry["content"])`. Images cannot get in. So the client-side observation in the cut comment stands — `HermesAgent.chat()` builds its prompt from `input_data.texts` and a captured frame has nowhere to go — but the fix is not "wait for Hermes to grow vision". It is a decision about which surface the bridge talks to, and that decision is not free. ### Why this is not simply unblocked The same surface split runs the other way for approvals: | Capability | `/v1/runs` (bridge is here) | `/v1/chat/completions` | |---|---|---| | Image input | no | **yes** | | Tool `args` | no | **yes** | | Stable `toolCallId` | no — the bridge FIFO-pairs by tool name | **yes** | | `approval.request` + `waiting_for_approval` | **yes** | **no** — no approval handling in that path at all | | Tool result payload | no | no — emitted nowhere | `_approval_notify` lives inside `_handle_runs` (`api_server.py:6250`). Migrating wholesale would trade approvals — already unblocked, already scoped at #19 — for images and args. That is not obviously a win. ### Actual blocked-by One of: 1. **`/v1/runs` accepts multimodal content** — an upstream change to `NousResearch/hermes-agent`, which is the honest version of the original reopen condition. 2. **A decided surface strategy** — chat-completions for image turns and runs for the rest, or a full migration that re-solves approvals. Real architecture work, tracked at `ollvt-hermes-bridge#19`. Until one lands there is still no path for a captured frame, so **nothing in this issue should be built yet**. Reopened because the blocker is now correctly identified and partly within our control, not because it is ready. Consequence for #9: screen-capture code stays parked rather than being deleted. That instruction is reverted there.

aiko

2026-07-29 23:13:35 +00:00

reopened this issue
added
ready-for-human
and removed
ready-for-agent
labels

aiko referenced this issue

2026-07-29 23:13:36 +00:00

Cut group sessions, camera, and the browser panel #9

aiko referenced this issue

2026-07-30 00:03:50 +00:00

画面共有 — a grant, not a panel #31

aiko commented

2026-07-30 00:03:50 +00:00

Author

Owner

Closing in favour of #31, rewritten against the ADRs.

This issue's premise changed with ADR-0001: Denpa is a Hermes gateway Platform adapter and the api_server HTTP surface is abandoned entirely. Patching the body would have left a trail of corrections on top of a stale audit block - the same pattern that got 承認 wrongly cut once already.

Decisions now live in docs/adr/ in the go-between repo (branch docs/hermes-platform-adrs). The replacement issue references them rather than restating them.

Closing in favour of #31, rewritten against the ADRs. This issue's premise changed with ADR-0001: Denpa is a Hermes gateway Platform adapter and the `api_server` HTTP surface is abandoned entirely. Patching the body would have left a trail of corrections on top of a stale audit block - the same pattern that got 承認 wrongly cut once already. Decisions now live in `docs/adr/` in the go-between repo (branch `docs/hermes-platform-adrs`). The replacement issue references them rather than restating them.

aiko closed this issue

2026-07-30 00:03:51 +00:00

aiko referenced this issue

2026-07-30 00:04:28 +00:00

Cut group sessions, camera, and the browser panel #9

aiko referenced this issue

2026-07-30 09:57:32 +00:00

Onboarding — six steps ending in a paired device #28

aiko referenced this issue from a commit

2026-07-30 19:17:59 +00:00

fix(onboarding): 姿 picks a Live2D model, not a dead persona bundle