She thinks for seconds and the dock says nothing true about it #83

Closed
opened 2026-08-01 14:02:41 +00:00 by aiko · 1 comment
Owner

Held a real voice conversation on 2026-07-31 and the operator's verdict on what was legible in the moment was: "I could only see tool calls in the logs. that's it."

That is not a treatment problem. It is that the two signals designed to say what she is doing during her think time are, respectively, a discarded frame and a state the client infers instead of reading.

Two halves

1. WAITING does not survive the think time. conversation-chain-start fires only once Hermes has produced the whole turn (single_conversation.py's send_conversation_start_signals runs after the finished text arrives, not when the turn is accepted), so the gap between the press and her first word is the entire think time. WAITING auto-returns to IDLE after 2s, so the dock reads 入力中 → 待機 → 送信中. The meter never changes — 待機 and 入力中 share the idle floor — so what actually differs is the word, the PTT key returning to 押して送信, and 割り込み un-dimming for a turn that is still in flight.

#58's close-out flagged this and asked for a decision. Decided 2026-08-01: hold WAITING until chain-start or a fault. The 2s is not a new timer being removed — it exists to un-stick a typing debounce, and a sent turn does not need un-sticking. Applying it to a sent turn was the mistake.

2. typing arrives on every turn and is thrown away. DenpaAdapter.send_typing and stop_typing are the PlatformAdapter methods Hermes' own gateway/platforms/base.py calls around every turn; AdapterHandler.handle_frame translates them and WebSocketHandler.send_to_device delivers them. The client has no case — websocket-handler.tsx's handleWebSocketMessage falls to default: and logs Unknown message type. Established by denpatou#32.

So the truthful signal for exactly the state half 1 is about is already on the socket, arriving, and being discarded — while the client infers the same state from aiState.

What is needed

  • WAITING holds from send until chain-start or a fault, rather than for 2s.
  • The client reads typing and uses it as the signal, rather than inferring the state. active: true is her working; active: false without a chain-start is the interesting case and wants a deliberate answer rather than falling through.
  • 割り込み stays dimmed and disabled for the whole of it, which is what #58 already decided for its two seconds.
  • The typing frame stops being logged as unknown.

Acceptance

  • After a send, the dock stays at 入力中 until she begins or something fails — no 待機 flicker on a slow turn
  • 割り込み is dim and unpressable for the whole think time, not the first 2s of it
  • The typing frame is consumed rather than warned about, and drives the state
  • A turn that fails during the think time leaves WAITING for the fault screen, not for IDLE
  • The 2s auto-return still applies to the typing-debounce case it was written for
  • Judged in a real conversation with a slow turn, not only in tests

denpa#58 (WAITING's treatment, and the call this settles), denpatou#32 (the dropped frame), denpa#59 (the fault screens WAITING hands off to)

Held a real voice conversation on 2026-07-31 and the operator's verdict on what was legible in the moment was: *"I could only see tool calls in the logs. that's it."* That is not a treatment problem. It is that the two signals designed to say what she is doing during her think time are, respectively, a discarded frame and a state the client infers instead of reading. ## Two halves **1. WAITING does not survive the think time.** `conversation-chain-start` fires only once Hermes has produced the *whole* turn (`single_conversation.py`'s `send_conversation_start_signals` runs after the finished text arrives, not when the turn is accepted), so the gap between the press and her first word is the entire think time. WAITING auto-returns to IDLE after 2s, so the dock reads 入力中 → 待機 → 送信中. The meter never changes — 待機 and 入力中 share the idle floor — so what actually differs is the word, the PTT key returning to 押して送信, and **割り込み un-dimming for a turn that is still in flight**. `#58`'s close-out flagged this and asked for a decision. **Decided 2026-08-01: hold WAITING until chain-start or a fault.** The 2s is not a new timer being removed — it exists to un-stick a *typing* debounce, and a sent turn does not need un-sticking. Applying it to a sent turn was the mistake. **2. `typing` arrives on every turn and is thrown away.** `DenpaAdapter.send_typing` and `stop_typing` are the `PlatformAdapter` methods Hermes' own `gateway/platforms/base.py` calls around every turn; `AdapterHandler.handle_frame` translates them and `WebSocketHandler.send_to_device` delivers them. The client has no case — `websocket-handler.tsx`'s `handleWebSocketMessage` falls to `default:` and logs `Unknown message type`. Established by [denpatou#32](https://git.aiko.works/aiko/denpatou/issues/32). So the truthful signal for exactly the state half 1 is about is already on the socket, arriving, and being discarded — while the client infers the same state from `aiState`. ## What is needed - WAITING holds from send until chain-start or a fault, rather than for 2s. - The client reads `typing` and uses it as the signal, rather than inferring the state. `active: true` is her working; `active: false` without a chain-start is the interesting case and wants a deliberate answer rather than falling through. - 割り込み stays dimmed and disabled for the whole of it, which is what `#58` already decided for its two seconds. - The typing frame stops being logged as unknown. ## Acceptance - [ ] After a send, the dock stays at 入力中 until she begins or something fails — no 待機 flicker on a slow turn - [ ] 割り込み is dim and unpressable for the whole think time, not the first 2s of it - [ ] The `typing` frame is consumed rather than warned about, and drives the state - [ ] A turn that fails during the think time leaves WAITING for the fault screen, not for IDLE - [ ] The 2s auto-return still applies to the typing-debounce case it was written for - [ ] Judged in a real conversation with a slow turn, not only in tests ## Related `denpa#58` (WAITING's treatment, and the call this settles), `denpatou#32` (the dropped frame), `denpa#59` (the fault screens WAITING hands off to)
Author
Owner

This was generated by AI during triage.

Merged locally as e2c069a (feature 2e60978, cross-seam fix af62cc6). Nothing pushed.

Acceptance

  • The dock stays at 入力中 after a send until she begins or something fails — ai-state-context.tsx grows a hold option and a pure waitingPush(current, options); the two sender surfaces (use-text-input.tsx's handleSend, transmission.ts's pttUp) pass it and the two keystroke surfaces do not.
  • 割り込み dim and unpressable for the whole think time — chrome-spec.ts's interruptIsDim reads aiState === WAITING, so it follows from the hold.
  • The typing frame is consumed and drives the state — services/typing.ts's pure typingStep(active, aiState), 13 cases plus a wiring scan. The producer was verified on the go-between's merged tree: adapter_handler.py's FRAME_TYPING branch sends {"type": "typing", "active": …} to the device, coercing an unsaid active to false, which is why the client reads === true.
  • A turn that fails during the think time leaves WAITING for the fault screen — and only now records one. Under the old timer a failure slower than two seconds arrived at IDLE and runFailureAfterMessage's turnIsOutstanding dropped it silently. Real improvement, not itself covered by a test.
  • The 2s still applies to the typing debounce it was written for
  • Judged in a real conversation with a slow turn. Not done.

active: false with no chain-start

Re-enters WAITING un-held, handing it to the same 2s return. Holding would strand the dock at 入力中 with no way back but a redial, since nothing else is coming for that turn; snapping to IDLE would flicker if a chain-start is racing in behind the stop, and the ordinary order is chain-start then stop. The 2s window is exactly the un-sticking that timer was written for.

What the cross-seam review found, and it is the reason this merged with #82

onPttUp draws the held WAITING synchronously, because a release has to feel immediate. Whether the press reached Rust is only knowable once the press chain settles — a short tap can release while mic_start is still opening the device. Before this issue, a press that never landed was swept up by the 2s return. With the hold, nothing swept it up, so a refused mic_start left the dock at 入力中 waiting for a chain-start no turn had earned, until the socket moved. #82 adds a brand-new way for that refusal to happen — startForeground throwing.

The obvious fix, refusing to draw WAITING unless the press is held, is wrong the same way: at release time a good press may still be opening, and that tap would then never show WAITING at all. Fixed instead by walking it back where the answer is final — pressNeverLanded in transmission.ts, reached from the release chain.

Per-issue review could not have seen this. It is what the batch's combined-diff pass is for.

Deliberately left

A turn that dies upstream with neither a chain-start nor a stop_typing still strands the hold until the socket moves. No second release path was added for it: that is the go-between's to answer, and inventing a client-side timeout would rebuild the two-second return this issue removed. The escape hatch is that a reconnect brings set-model-and-conf, which sets IDLE — unverified that the go-between re-sends it on every reconnect.

Also unverified: that a typing frame arrives on this route in practice. denpatou#32 established the producer by reading; no gateway was run.

Gates

npm test 941 across 73 files (was 917/70), npm run typecheck exits 0, cargo test 28. npm run lint is broken repo-wide — the airbnb config it references cannot be resolved — which predates this work.

> *This was generated by AI during triage.* Merged locally as `e2c069a` (feature `2e60978`, cross-seam fix `af62cc6`). Nothing pushed. ## Acceptance - [x] The dock stays at 入力中 after a send until she begins or something fails — `ai-state-context.tsx` grows a `hold` option and a pure `waitingPush(current, options)`; the two sender surfaces (`use-text-input.tsx`'s `handleSend`, `transmission.ts`'s `pttUp`) pass it and the two keystroke surfaces do not. - [x] 割り込み dim and unpressable for the whole think time — `chrome-spec.ts`'s `interruptIsDim` reads `aiState === WAITING`, so it follows from the hold. - [x] The `typing` frame is consumed and drives the state — `services/typing.ts`'s pure `typingStep(active, aiState)`, 13 cases plus a wiring scan. The producer was verified on the go-between's merged tree: `adapter_handler.py`'s `FRAME_TYPING` branch sends `{"type": "typing", "active": …}` to the device, coercing an unsaid `active` to false, which is why the client reads `=== true`. - [x] A turn that fails during the think time leaves WAITING for the fault screen — **and only now records one.** Under the old timer a failure slower than two seconds arrived at IDLE and `runFailureAfterMessage`'s `turnIsOutstanding` dropped it silently. Real improvement, not itself covered by a test. - [x] The 2s still applies to the typing debounce it was written for - [ ] **Judged in a real conversation with a slow turn.** Not done. ## `active: false` with no chain-start Re-enters WAITING **un-held**, handing it to the same 2s return. Holding would strand the dock at 入力中 with no way back but a redial, since nothing else is coming for that turn; snapping to IDLE would flicker if a chain-start is racing in behind the stop, and the ordinary order is chain-start *then* stop. The 2s window is exactly the un-sticking that timer was written for. ## What the cross-seam review found, and it is the reason this merged with `#82` `onPttUp` draws the held WAITING synchronously, because a release has to feel immediate. Whether the press reached Rust is only knowable once the press chain settles — a short tap can release while `mic_start` is still opening the device. **Before this issue, a press that never landed was swept up by the 2s return. With the hold, nothing swept it up**, so a refused `mic_start` left the dock at 入力中 waiting for a chain-start no turn had earned, until the socket moved. `#82` adds a brand-new way for that refusal to happen — `startForeground` throwing. The obvious fix, refusing to draw WAITING unless the press is held, is wrong the same way: at release time a *good* press may still be opening, and that tap would then never show WAITING at all. Fixed instead by walking it back where the answer is final — `pressNeverLanded` in `transmission.ts`, reached from the release chain. Per-issue review could not have seen this. It is what the batch's combined-diff pass is for. ## Deliberately left A turn that dies upstream with **neither** a chain-start nor a `stop_typing` still strands the hold until the socket moves. No second release path was added for it: that is the go-between's to answer, and inventing a client-side timeout would rebuild the two-second return this issue removed. The escape hatch is that a reconnect brings `set-model-and-conf`, which sets IDLE — unverified that the go-between re-sends it on every reconnect. Also unverified: that a `typing` frame arrives on this route in practice. `denpatou#32` established the producer by reading; no gateway was run. ## Gates `npm test` 941 across 73 files (was 917/70), `npm run typecheck` exits 0, `cargo test` 28. `npm run lint` is broken repo-wide — the `airbnb` config it references cannot be resolved — which predates this work.
aiko closed this issue 2026-08-01 15:12:01 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
aiko/denpa#83
No description provided.