She stops listening the instant you pause for breath #94

New issue

Open

opened 2026-08-17 11:51:35 +00:00 by aiko · 0 comments

aiko commented

2026-08-17 11:51:35 +00:00

Owner

She stops listening the instant you pause for breath.

Reported 2026-08-17, on the first real use of 呼びかけ受信 (#90): the wake phrase works, and then the sentence after it gets cut off partway through.

Why

Detector::observe (src-tauri/src/audio/engine.rs) ends an utterance on the first frame whose probability falls below the negative threshold:

if self.speaking && probability < self.thresholds.negative {
  self.speaking = false;
  self.hold.reset();
  return (Some(Edge::End), 0.0);
}

A silero frame is 512 samples, which at 16 kHz is 32 ms. So a single 32 ms dip closes the utterance and transmits whatever had accumulated. Thirty-two milliseconds is shorter than the gap between two ordinary words, shorter than an unvoiced stop, and far shorter than someone drawing breath mid-sentence. The detector is not mistuned; it has no hangover at all.

The knob for this already exists and does nothing

VADSettings.redemptionFrames is still in the settings screen, still stored, still defaulting to 35 — and inert since capture moved to Rust. vad-context.tsx:48 says so outright:

Kept for the settings screen and for stored preferences, but inert since capture moved to Rust: the detector there ends an utterance by falling below the negative threshold rather than by counting redemption frames.

That comment was written to stop somebody losing an afternoon to it. This issue is that afternoon arriving: the control the operator would reach for is exactly the right one, and it is disconnected.

What to do

Require N consecutive sub-threshold frames before emitting Edge::End, and reset the counter on any frame back above the threshold. That is the redemption the JS VAD had, in the place the detector now lives.

Wire redemptionFrames through mic_start alongside the two thresholds and wake_threshold (#90 just did this for the wake threshold and is the pattern to copy). The stored default of 35 frames is 1.12 s of silence before she decides you have finished, which is a reasonable starting point precisely because it is the number the setting has claimed all along.

Two things to get right:

The pre-roll and the buffer must not lose the redeemed audio. Frames below threshold that are later redeemed are part of the utterance -- they are the pause inside the sentence. Dropping them and rejoining leaves a clipped recording that transcribes badly.
MAX_UTTERANCE_SAMPLES still applies. A longer hangover makes a stuck-open detector likelier, and the 60 s ceiling is what stops that growing the buffer until the process dies.

#90 — the settings plumbing pattern, and where a live control belongs
#89 — the gate, which is downstream of this and unaffected: it decides whether to send an utterance, not where the utterance ends
#43 — speech start and end meaning the same thing in both modes

She stops listening the instant you pause for breath. Reported 2026-08-17, on the first real use of 呼びかけ受信 (`#90`): the wake phrase works, and then the sentence after it gets cut off partway through. ## Why `Detector::observe` (`src-tauri/src/audio/engine.rs`) ends an utterance on the **first** frame whose probability falls below the negative threshold: ```rust if self.speaking && probability < self.thresholds.negative { self.speaking = false; self.hold.reset(); return (Some(Edge::End), 0.0); } ``` A silero frame is 512 samples, which at 16 kHz is **32 ms**. So a single 32 ms dip closes the utterance and transmits whatever had accumulated. Thirty-two milliseconds is shorter than the gap between two ordinary words, shorter than an unvoiced stop, and far shorter than someone drawing breath mid-sentence. The detector is not mistuned; it has no hangover at all. ## The knob for this already exists and does nothing `VADSettings.redemptionFrames` is still in the settings screen, still stored, still defaulting to 35 — and inert since capture moved to Rust. `vad-context.tsx:48` says so outright: > Kept for the settings screen and for stored preferences, but **inert since capture moved to Rust**: the detector there ends an utterance by falling below the negative threshold rather than by counting redemption frames. That comment was written to stop somebody losing an afternoon to it. This issue is that afternoon arriving: the control the operator would reach for is exactly the right one, and it is disconnected. ## What to do Require **N consecutive** sub-threshold frames before emitting `Edge::End`, and reset the counter on any frame back above the threshold. That is the redemption the JS VAD had, in the place the detector now lives. Wire `redemptionFrames` through `mic_start` alongside the two thresholds and `wake_threshold` (`#90` just did this for the wake threshold and is the pattern to copy). The stored default of 35 frames is 1.12 s of silence before she decides you have finished, which is a reasonable starting point precisely because it is the number the setting has claimed all along. Two things to get right: - **The pre-roll and the buffer must not lose the redeemed audio.** Frames below threshold that are later redeemed are part of the utterance -- they are the pause *inside* the sentence. Dropping them and rejoining leaves a clipped recording that transcribes badly. - **`MAX_UTTERANCE_SAMPLES` still applies.** A longer hangover makes a stuck-open detector likelier, and the 60 s ceiling is what stops that growing the buffer until the process dies. ## Related - `#90` — the settings plumbing pattern, and where a live control belongs - `#89` — the gate, which is downstream of this and unaffected: it decides *whether* to send an utterance, not where the utterance ends - `#43` — speech start and end meaning the same thing in both modes

aiko added the

ready-for-agent

labels

2026-08-17 11:51:35 +00:00