Press-to-talk transmits nothing on a short press #33

New issue

Closed

opened 2026-07-30 07:54:03 +00:00 by aiko · 2 comments

aiko commented

2026-07-30 07:54:03 +00:00

Owner

Press-to-talk in the mobile dock cycles its states correctly and transmits nothing. The control looks right, the tests pass, and the feature does not work.

#12 built the dock's PTT key to drive the mic through VADContext — startMic() on pointerdown, stopMic() on pointerup. But the only path that actually sends audio upstream is VAD's own onSpeechEnd, and stopMic() destroys the VAD instance without flushing (src/renderer/src/context/vad-context.tsx:322).

So a release before VAD detects speech-end sends nothing at all. A short utterance, or any press where VAD does not fire in time, is silently discarded.

This was found and reported by the agent that implemented #12, not by a user hitting it — it is in .denpa-work/QUESTIONS.md. It was correctly judged out of scope for a chrome issue: fixing it needs a recorder in the mic layer, not in the chrome.

What is needed

A capture path that does not depend on VAD's speech-end firing. On release, whatever was recorded during the press should go up as denpa.voice regardless of VAD's opinion — press-to-talk is an explicit human signal and should not be second-guessed by a voice-activity heuristic.

Acceptance

Pressing and releasing quickly still transmits the audio captured during the press.
A release mid-utterance transmits what was captured, without waiting for speech-end.
Hands-free (VAD-driven) capture keeps working unchanged — this adds a path, it does not replace one.
A test that fails if a press-release shorter than VAD's speech-end window transmits nothing.

Notes

Reads as done on the board today, which is the real hazard here.

Press-to-talk in the mobile dock cycles its states correctly and transmits nothing. The control looks right, the tests pass, and the feature does not work. `#12` built the dock's PTT key to drive the mic through `VADContext` — `startMic()` on pointerdown, `stopMic()` on pointerup. But the only path that actually sends audio upstream is VAD's own `onSpeechEnd`, and `stopMic()` destroys the VAD instance without flushing (`src/renderer/src/context/vad-context.tsx:322`). **So a release before VAD detects speech-end sends nothing at all.** A short utterance, or any press where VAD does not fire in time, is silently discarded. This was found and reported by the agent that implemented `#12`, not by a user hitting it — it is in `.denpa-work/QUESTIONS.md`. It was correctly judged out of scope for a chrome issue: fixing it needs a recorder in the mic layer, not in the chrome. ## What is needed A capture path that does not depend on VAD's speech-end firing. On release, whatever was recorded during the press should go up as `denpa.voice` regardless of VAD's opinion — press-to-talk is an explicit human signal and should not be second-guessed by a voice-activity heuristic. ## Acceptance - Pressing and releasing quickly still transmits the audio captured during the press. - A release mid-utterance transmits what was captured, without waiting for speech-end. - Hands-free (VAD-driven) capture keeps working unchanged — this adds a path, it does not replace one. - A test that fails if a press-release shorter than VAD's speech-end window transmits nothing. ## Notes Reads as done on the board today, which is the real hazard here.

aiko added the

ready-for-agent

label

2026-07-30 07:54:03 +00:00

aiko commented

2026-07-30 08:37:55 +00:00

Author

Owner

Superseded approach, by human decision on 2026-07-30.

Do not fix this by adding a recorder beside the existing JS VAD. All microphone capture and voice-activity detection are moving out of the webview into Rust — see aiko/denpa#35.

The root cause named here is the same either way: press-to-talk transmits nothing because VAD owns the mic and \stopMic()\ destroys the instance without flushing. That fix now arrives as part of the Rust capture path, where a release can flush what it recorded.

Leaving this issue open as the bug of record; #35 carries the implementation.

**Superseded approach, by human decision on 2026-07-30.** Do not fix this by adding a recorder beside the existing JS VAD. All microphone capture and voice-activity detection are moving out of the webview into Rust — see aiko/denpa#35. The root cause named here is the same either way: press-to-talk transmits nothing because VAD owns the mic and \stopMic()\ destroys the instance without flushing. That fix now arrives as part of the Rust capture path, where a release can flush what it recorded. Leaving this issue open as the bug of record; #35 carries the implementation.

aiko commented

2026-07-30 16:57:08 +00:00

Author

Owner

Fixed and verified. Closing.

A press now transmits what it captured whether or not the detector ever fired, because the press owns its own buffer in Rust and the release flushes it:

src-tauri/src/audio/desktop.rs — Signal::PressStart seeds a buffer from the pre-roll, Signal::PressRelease emits it as speech-end with from_press: true, and the detector abandons its own utterance on release rather than emitting the same audio a second time.
Half a second of pre-roll rides ahead of every press, so the attack of the first word survives reaction time — this issue's fix, plus the thing that would have made it sound broken anyway.
An empty press still emits, so the renderer can tell a press that captured nothing from an event that never arrived.
use-receiver-chrome.ts chains press and release on one promise, so a tap released before the microphone finishes opening cannot leave a press that never ends.

Observed in a real conversation on 2026-07-30:

voice up:  46720 samples,  2.92s, peak 0.0668
voice up: 217760 samples, 13.61s, peak 0.0761

Landed on branch issue/denpa-35-mic-capture-into-rust, commits 0da3018 and bffad16. Not merged.

Note the mechanism is platform-independent — the press buffer and the release flush live in the shared worker, not in the desktop capture backend — so Android inherits the behaviour when its capture path lands under #35, rather than needing this fixed again.

**Fixed and verified.** Closing. A press now transmits what it captured whether or not the detector ever fired, because the press owns its own buffer in Rust and the release flushes it: - `src-tauri/src/audio/desktop.rs` — `Signal::PressStart` seeds a buffer from the pre-roll, `Signal::PressRelease` emits it as `speech-end` with `from_press: true`, and the detector **abandons** its own utterance on release rather than emitting the same audio a second time. - Half a second of pre-roll rides ahead of every press, so the attack of the first word survives reaction time — this issue's fix, plus the thing that would have made it sound broken anyway. - An empty press still emits, so the renderer can tell a press that captured nothing from an event that never arrived. - `use-receiver-chrome.ts` chains press and release on one promise, so a tap released before the microphone finishes opening cannot leave a press that never ends. Observed in a real conversation on 2026-07-30: ``` voice up: 46720 samples, 2.92s, peak 0.0668 voice up: 217760 samples, 13.61s, peak 0.0761 ``` Landed on branch `issue/denpa-35-mic-capture-into-rust`, commits `0da3018` and `bffad16`. Not merged. **Note the mechanism is platform-independent** — the press buffer and the release flush live in the shared worker, not in the desktop capture backend — so Android inherits the behaviour when its capture path lands under `#35`, rather than needing this fixed again.

aiko closed this issue

2026-07-30 16:57:08 +00:00

aiko referenced this issue from a commit

2026-07-30 17:28:45 +00:00

feat(audio): capture and detect speech in Rust, not the webview

aiko referenced this issue from a commit

2026-07-30 17:28:45 +00:00

feat(audio): drive the microphone from Rust, and let the operator choose it