PRD: Kurageyomi — sentence mining from Japanese Jellyfin subtitles #1

New issue

Open

opened 2026-08-09 20:04:28 +00:00 by aiko · 1 comment

aiko commented 2026-08-09 20:04:28 +00:00

Owner

Copy link

Kurageyomi — sentence mining from Japanese Jellyfin subtitles

Problem Statement

I watch Japanese shows on my own Jellyfin server to study the language, and the words I most
want to learn are the ones I meet while watching. Right now there is no way to act on them in
the moment. If I want a word, I have to stop the episode, get to another device, retype
something I only half-heard into a dictionary, guess at its dictionary form, and then rebuild
the sentence and the moment by hand into an Anki card. It is enough friction that I mostly
don't bother, so the words I encounter while watching are exactly the words I never learn.

Existing sentence-mining setups solve this for local video players, but my library lives on a
Jellyfin server and I watch it on whatever screen is convenient. Nothing bridges those.

Solution

A Jellyfin plugin that turns my phone into a mining surface for whatever is currently playing.

The plugin serves a web page over the tailnet. When an episode is playing on any of my
clients, the phone shows the Japanese Cue currently on screen, following the Playhead of that
Playback Session. Tapping an unknown word pauses playback, runs a Yomichan-style lookup —
deinflecting the Surface Form to a Headword and showing the dictionary entry — and saves it as
a Capture with the whole Cue for context. Dismissing the lookup resumes the show.

Captures accumulate in a Mining Queue that lives on the server, so mining never depends on
Anki being open or reachable. Later, an Export takes the whole queue, cuts an audio Clip and a
still Frame for each Capture straight from the source file with ffmpeg, and pushes finished
Notes into Anki over the tailnet.

The result: seeing an unknown word and having a card with audio, an image, the sentence, and a
definition becomes one tap, taken without leaving the sofa.

User Stories

1. As a learner, I want the phone to show the Japanese Cue currently on screen, so that I can
   read along with what I am hearing.
2. As a learner, I want the phone to find the playing Playback Session by itself, so that I do
   not configure anything before starting an episode.
3. As a learner, I want to pick from a list when more than one of my clients is playing, so
   that the phone follows the right screen.
4. As a learner, I want the phone to remember the Playback Session I picked last time, so that
   the common case needs no interaction.
5. As a learner, I want the displayed Cue to stay in step with the video between the client's
   position reports, so that the text on my phone matches the audio rather than lagging behind
   it.
6. As a learner, I want the phone to correct itself immediately when I pause or seek, so that a
   jump in the show does not leave stale text in my hand.
7. As a learner, I want the phone to keep the screen awake while an episode is playing, so that
   I am not unlocking it every thirty seconds.
8. As a learner, I want to tap any word in the Cue, so that I can look it up without typing.
9. As a learner, I want a tap to resolve the longest sensible word starting at that point, so
   that tapping the first character of a compound gives me the compound.
10. As a learner, I want inflected words to resolve to their dictionary form, so that tapping
    食べさせられなかった finds 食べる.
11. As a learner, I want to see the chain of inflections that were undone, so that I understand
    why the word looks the way it does on screen.
12. As a learner, I want to see several candidate entries when a lookup is ambiguous, so that I
    can choose the sense that actually fits the scene.
13. As a learner, I want playback to pause automatically when I tap a word, so that the show
    does not run on while I read.
14. As a learner, I want playback to resume when I dismiss the lookup, so that returning to the
    episode costs nothing.
15. As a learner, I want to dismiss a lookup without saving anything, so that idle curiosity
    does not fill my deck.
16. As a learner, I want to save a looked-up word as a Capture with one tap, so that mining
    stays fast enough to do repeatedly in a single scene.
17. As a learner, I want the Capture to record the Surface Form as it appeared, so that I can
    later see the word in the shape the show used.
18. As a learner, I want the Capture to record the whole Cue, so that the sentence and not just
    the word is what I study.
19. As a learner, I want the Capture to record the definition as it read at the time, so that a
    later dictionary update cannot silently change what I learned.
20. As a learner, I want the Capture to record which episode and moment it came from, so that
    Clips and Frames can be cut later and so I can find the scene again.
21. As a learner, I want the app to tell me when I have already mined a Headword, so that I do
    not build duplicate cards for the same word across episodes.
22. As a learner, I want that duplicate check to work while Anki is closed, so that mining is
    never blocked by something being unreachable.
23. As a learner, I want to reveal the English subtitle for the current moment with a tap, so
    that I have a fallback when a line is grammatically rather than lexically hard.
24. As a learner, I want the English hidden by default, so that I am not reading it by reflex
    instead of the Japanese.
25. As a learner, I want to choose which English track is used when a show has several, so that
    I am not stuck with signs-and-songs or a commentary track.
26. As a learner, I want styling and positioning tags stripped from Cues, so that ASS subtitles
    read as text rather than as markup.
27. As a learner, I want ruby furigana in the source track handled rather than dumped inline,
    so that a styled subtitle does not become unreadable.
28. As a learner, I want a clear message when a show has only image-based subtitles, so that I
    understand why mining is unavailable instead of seeing a blank screen.
29. As a learner, I want my Captures to survive clearing my phone's browser data, so that a
    session's work is not one tap from being lost.
30. As a learner, I want to see how many Captures are waiting, so that I know whether an Export
    is worth doing.
31. As a learner, I want to trigger an Export deliberately, so that cards appear in Anki when I
    choose rather than as a side effect of watching.
32. As a learner, I want Export to cut an audio Clip of each Cue, so that my cards test
    listening and not just reading.
33. As a learner, I want Export to capture a still Frame from each Cue, so that the card recalls
    the scene it came from.
34. As a learner, I want Clips padded slightly beyond the Cue's exact boundaries, so that the
    audio does not clip the first or last mora.
35. As a learner, I want the note type created for me on first Export, so that I do not hand-build
    a note type in Anki before I can use the app.
36. As a learner, I want Export to skip Headwords already exported, so that repeated Export runs
    do not duplicate cards.
37. As a learner, I want to be told which Captures failed to Export and why, so that a partial
    failure is visible rather than silent.
38. As a learner, I want failed Captures to stay in the Mining Queue, so that a retry picks them
    up rather than losing them.
39. As a learner, I want Export to be safe to run twice, so that a network wobble mid-export does
    not force me to reason about what already landed.
40. As a learner, I want extraction to happen at Export rather than while I watch, so that mining
    a dense scene does not put ffmpeg jobs on a server that is already transcoding for me.
41. As a learner, I want dictionary attribution shown wherever definitions appear, so that the
    EDRDG licence is honoured.
42. As a learner, I want the dictionary available offline after first load, so that lookups do
    not depend on anything but the phone.
43. As a learner, I want to install the page to my home screen, so that it opens like an app
    rather than a browser tab.
44. As a learner, I want the app to use my existing Jellyfin login, so that there is no second
    account or token to manage.
45. As a learner, I want to configure where Anki lives and which deck receives cards, so that
    Export goes somewhere I chose.
46. As a learner, I want to see whether Anki is currently reachable before I Export, so that a
    failure is predicted rather than discovered.
47. As a learner, I want the plugin to leave Jellyfin working normally when the app is idle, so
    that a study tool never degrades the media server for anyone else using it.

Implementation Decisions

Everything below is settled by the design session recorded in CONTEXT.md and ADRs 0001–0004.

Shape

The whole app is a Jellyfin plugin (ADR 0001). It serves the phone UI as a plugin web asset,
same-origin, authenticated by the Jellyfin token the phone already holds. No sidecar service,
no second auth system, no CORS.

The phone side is an installable PWA, not a native shell. A Tauri wrapper was considered and
deferred: it would re-introduce the seams the plugin decision removes (bundled assets, its own
token) to buy protection against risks — chiefly IndexedDB eviction — that may never
materialise. Wrapping a working PWA later is cheap.

Target Jellyfin 10.11 (dionysus runs 10.11.11), not the 10.9 the official plugin template
pins. Jellyfin runs in a container, so ffmpeg comes from the server's own media encoder and
item paths are already container-relative.

Division of labour

C# handles sessions, media, storage, and export. The browser handles all Japanese. This split
is deliberate — the mature deinflection code is TypeScript, and there is no reason to port it.

Plugin modules

• CueTrack — takes a raw subtitle track and returns normalized Cues: start, end, and text
  with styling, positioning, and ruby markup removed. Jellyfin types do not appear at its
  boundary. Text tracks only.
• SessionWatcher — resolves which Playback Sessions belong to a user and exposes each one's
  Playhead, item, and stream indices.
• PlaybackRemote — pause, unpause, and seek a Playback Session.
• MiningQueue — persistence for Captures, in SQLite under the plugin data dir. Exposes add,
  list, drain, and the Known Snapshot. Plugin configuration is XML and is explicitly not used
  for this.
• MediaExtractor — given an item and a Cue's span, produces a Clip and a Frame. The
  encoder is injected; the module's real logic is argument construction — padding, seek
  strategy, and where in the Cue the Frame is taken.
• AnkiExporter — turns Captures into Notes. Owns note-type creation, media upload,
  duplicate checking, and partial-failure reporting.

Phone modules

• Deinflector — Surface Form to candidate Headwords with the reason chain. Lifted from
  10ten Japanese Reader (GPL-3.0) along with its existing tests, rather than written fresh.
• DictionaryIndex — the full jmdict-eng build bundled as a plugin asset, loaded once into
  IndexedDB, queried by longest match from a tap offset. No morphological tokenizer exists in
  this system (ADR 0002); lookup is tap-driven longest-match plus deinflection, which is why
  none is needed.
• PlayheadClock — combines the last reported position, wall-clock elapsed time, and paused
  state into a current position, and from that selects the current Cue.
• SessionFeed — subscribes to the server's session push and turns those messages into
  playback-state events.

Contracts and behaviour

• Playhead cadence. Clients report progress on a 10-second throttle while playing, but
  report immediately on pause, unpause, and any seek beyond a ~5-second drift. The phone
  therefore interpolates between reports and resyncs on every push; it must never assume a
  report is fresh.
• Auto-pause. Tapping a word issues a pause to the Playback Session; dismissing issues an
  unpause. Both produce an immediate progress report, which resyncs the Playhead as a side
  effect.
• Capture contents. Headword, Surface Form, the full Cue text, Cue start and end, item and
  media-source identity, the chosen dictionary entry as it read at capture time, and the
  deinflection reason chain.
• Note fields. Headword, Reading, Surface Form, Cue, Definition, Clip, Frame, Source. A
  fixed note type is created on first Export — the Anki collection is starting empty, so there
  is nothing to map onto and no migration to consider.
• Known Snapshot. Built from this app's own exported Notes only. The collection is fresh,
  so collection-wide indexing buys nothing yet; it is an additive change later if this is ever
  shared.
• Export transport. Server-to-server HTTP to AnkiConnect on dex-win over the tailnet
  (ADR 0003). Because it is not a browser call, CORS never applies.
• Export is idempotent and resumable. Successful Captures leave the queue; failed ones stay
  with their error, and a second run retries only those.

Security

AnkiConnect ships unauthenticated (apiKey defaults to none) and bound to loopback. Reaching
it from the plugin requires binding it beyond loopback — bind specifically to the Tailscale
interface address, never 0.0.0.0, and set an apiKey. AnkiConnect can read and modify the
entire collection; tailnet plus apiKey is acceptable, LAN or public is not.

Licensing

GPL-3.0, forced by the lifted deinflection table and consistent with Jellyfin's own GPL-3.0
plugin template (ADR 0004). JMdict's EDRDG licence requires visible attribution, which is a
product requirement rendered in the lookup UI, not a README line.

Testing Decisions

A good test here asserts external behaviour at a module's boundary — text in, Cues out;
Captures in, Anki calls out — and never reaches for internals. Tests should survive a rewrite
of a module's insides. Fakes stand in for Jellyfin's encoder and for AnkiConnect; nothing in
the suite requires a running Jellyfin, a running Anki, or a real media file beyond small
fixtures.

There is no prior art: this is a greenfield repo. Per standing practice, new logic gets tests
regardless of repo precedent, so the suite starts with this work rather than being retrofitted.

Tested — the six deep modules:

• CueTrack — tag stripping, ruby handling, timing conversion, malformed and empty tracks,
  overlapping cues, and image-track rejection.
• MiningQueue — add and drain round-trips, Known Snapshot contents, durability across
  restart, and behaviour when draining an empty queue.
• MediaExtractor — argument construction against a fake encoder: padding applied to Cue
  boundaries, seek behaviour, Frame position within the Cue, and handling of a Cue that starts
  at zero or runs to the end of the file.
• AnkiExporter — against a fake AnkiConnect: note-type creation on first run, media upload,
  duplicate skipping, partial failure leaving the right Captures queued, and a repeated run
  being a no-op.
• DictionaryIndex — longest-match resolution from a tap offset, ranking of candidates,
  behaviour at the end of a Cue, and lookups that match nothing.
• PlayheadClock — interpolation between reports, paused state freezing the clock, a seek
  arriving mid-interpolation, cue selection at exact boundaries, and a stale report that never
  gets superseded.

Deinflector arrives with its own test suite from 10ten; those tests come across with the
code rather than being rewritten.

Not tested: SessionWatcher, PlaybackRemote, and SessionFeed. All three are thin pass-throughs
to Jellyfin APIs where a test would assert the mock rather than any behaviour of ours. If
server upgrades start breaking them — a live risk given the rebuild-per-major consequence in
ADR 0001 — that judgement gets revisited.

Out of Scope

• Image-based subtitles. PGS and VOBSUB are bitmaps; supporting them means OCR. Shows with
  only image tracks get an explanatory message.
• Automatic highlighting of unknown words. Rejected outright, and it is what makes the
  absence of a tokenizer possible (ADR 0002). Noticing your own gaps is the study activity.
• A known-words model. The Known Snapshot is a duplicate check over our own Notes, not a
  model of what the learner knows.
• Frequency lists and word grading.
• Reviewing the Mining Queue before Export. Everything exports; Anki's browser is where
  mistakes get deleted.
• Editing Captures. No Cue-boundary adjustment, no definition editing.
• Indexing the existing Anki collection. Nothing to index yet. Additive later.
• A native Android app. Deferred, with the migration path deliberately kept cheap.
• AnkiDroid direct-add, and any export path that does not go through AnkiConnect.
• Generating .apkg files.
• Multi-user support. Personal tool; no per-user isolation beyond what Jellyfin already
  enforces on sessions.
• Languages other than Japanese, and dictionaries other than JMdict.
• Publishing to a public Jellyfin plugin repository. If that ever happens, the repo takes
  the jellyfin-plugin- prefix and the EDRDG commercial-use restriction needs a second look.

Further Notes

The riskiest assumption in the whole design is that a 10-second position report plus
client-side interpolation feels tight enough to read along with. Everything else is
conventional work; that one is a feel question no amount of planning settles. The cheapest
possible slice — plugin skeleton, one live Cue rendering on the phone, no dictionary, no
mining — tests it, and should come first.

Open questions deliberately left for implementation, none of them architectural: how much
padding a Clip gets and whether it may extend into an adjacent Cue; exactly where in a Cue the
Frame is taken; how English tracks are chosen when a show has several; and how partial Export
failures are surfaced in the UI.

dionysus currently reports a Tailscale DNS health warning. Worth ruling out before debugging
any connectivity problem between the plugin and dex-win.

# Kurageyomi — sentence mining from Japanese Jellyfin subtitles ## Problem Statement I watch Japanese shows on my own Jellyfin server to study the language, and the words I most want to learn are the ones I meet while watching. Right now there is no way to act on them in the moment. If I want a word, I have to stop the episode, get to another device, retype something I only half-heard into a dictionary, guess at its dictionary form, and then rebuild the sentence and the moment by hand into an Anki card. It is enough friction that I mostly don't bother, so the words I encounter while watching are exactly the words I never learn. Existing sentence-mining setups solve this for local video players, but my library lives on a Jellyfin server and I watch it on whatever screen is convenient. Nothing bridges those. ## Solution A Jellyfin plugin that turns my phone into a mining surface for whatever is currently playing. The plugin serves a web page over the tailnet. When an episode is playing on any of my clients, the phone shows the Japanese Cue currently on screen, following the Playhead of that Playback Session. Tapping an unknown word pauses playback, runs a Yomichan-style lookup — deinflecting the Surface Form to a Headword and showing the dictionary entry — and saves it as a Capture with the whole Cue for context. Dismissing the lookup resumes the show. Captures accumulate in a Mining Queue that lives on the server, so mining never depends on Anki being open or reachable. Later, an Export takes the whole queue, cuts an audio Clip and a still Frame for each Capture straight from the source file with ffmpeg, and pushes finished Notes into Anki over the tailnet. The result: seeing an unknown word and having a card with audio, an image, the sentence, and a definition becomes one tap, taken without leaving the sofa. ## User Stories 1. As a learner, I want the phone to show the Japanese Cue currently on screen, so that I can read along with what I am hearing. 2. As a learner, I want the phone to find the playing Playback Session by itself, so that I do not configure anything before starting an episode. 3. As a learner, I want to pick from a list when more than one of my clients is playing, so that the phone follows the right screen. 4. As a learner, I want the phone to remember the Playback Session I picked last time, so that the common case needs no interaction. 5. As a learner, I want the displayed Cue to stay in step with the video between the client's position reports, so that the text on my phone matches the audio rather than lagging behind it. 6. As a learner, I want the phone to correct itself immediately when I pause or seek, so that a jump in the show does not leave stale text in my hand. 7. As a learner, I want the phone to keep the screen awake while an episode is playing, so that I am not unlocking it every thirty seconds. 8. As a learner, I want to tap any word in the Cue, so that I can look it up without typing. 9. As a learner, I want a tap to resolve the longest sensible word starting at that point, so that tapping the first character of a compound gives me the compound. 10. As a learner, I want inflected words to resolve to their dictionary form, so that tapping 食べさせられなかった finds 食べる. 11. As a learner, I want to see the chain of inflections that were undone, so that I understand why the word looks the way it does on screen. 12. As a learner, I want to see several candidate entries when a lookup is ambiguous, so that I can choose the sense that actually fits the scene. 13. As a learner, I want playback to pause automatically when I tap a word, so that the show does not run on while I read. 14. As a learner, I want playback to resume when I dismiss the lookup, so that returning to the episode costs nothing. 15. As a learner, I want to dismiss a lookup without saving anything, so that idle curiosity does not fill my deck. 16. As a learner, I want to save a looked-up word as a Capture with one tap, so that mining stays fast enough to do repeatedly in a single scene. 17. As a learner, I want the Capture to record the Surface Form as it appeared, so that I can later see the word in the shape the show used. 18. As a learner, I want the Capture to record the whole Cue, so that the sentence and not just the word is what I study. 19. As a learner, I want the Capture to record the definition as it read at the time, so that a later dictionary update cannot silently change what I learned. 20. As a learner, I want the Capture to record which episode and moment it came from, so that Clips and Frames can be cut later and so I can find the scene again. 21. As a learner, I want the app to tell me when I have already mined a Headword, so that I do not build duplicate cards for the same word across episodes. 22. As a learner, I want that duplicate check to work while Anki is closed, so that mining is never blocked by something being unreachable. 23. As a learner, I want to reveal the English subtitle for the current moment with a tap, so that I have a fallback when a line is grammatically rather than lexically hard. 24. As a learner, I want the English hidden by default, so that I am not reading it by reflex instead of the Japanese. 25. As a learner, I want to choose which English track is used when a show has several, so that I am not stuck with signs-and-songs or a commentary track. 26. As a learner, I want styling and positioning tags stripped from Cues, so that ASS subtitles read as text rather than as markup. 27. As a learner, I want ruby furigana in the source track handled rather than dumped inline, so that a styled subtitle does not become unreadable. 28. As a learner, I want a clear message when a show has only image-based subtitles, so that I understand why mining is unavailable instead of seeing a blank screen. 29. As a learner, I want my Captures to survive clearing my phone's browser data, so that a session's work is not one tap from being lost. 30. As a learner, I want to see how many Captures are waiting, so that I know whether an Export is worth doing. 31. As a learner, I want to trigger an Export deliberately, so that cards appear in Anki when I choose rather than as a side effect of watching. 32. As a learner, I want Export to cut an audio Clip of each Cue, so that my cards test listening and not just reading. 33. As a learner, I want Export to capture a still Frame from each Cue, so that the card recalls the scene it came from. 34. As a learner, I want Clips padded slightly beyond the Cue's exact boundaries, so that the audio does not clip the first or last mora. 35. As a learner, I want the note type created for me on first Export, so that I do not hand-build a note type in Anki before I can use the app. 36. As a learner, I want Export to skip Headwords already exported, so that repeated Export runs do not duplicate cards. 37. As a learner, I want to be told which Captures failed to Export and why, so that a partial failure is visible rather than silent. 38. As a learner, I want failed Captures to stay in the Mining Queue, so that a retry picks them up rather than losing them. 39. As a learner, I want Export to be safe to run twice, so that a network wobble mid-export does not force me to reason about what already landed. 40. As a learner, I want extraction to happen at Export rather than while I watch, so that mining a dense scene does not put ffmpeg jobs on a server that is already transcoding for me. 41. As a learner, I want dictionary attribution shown wherever definitions appear, so that the EDRDG licence is honoured. 42. As a learner, I want the dictionary available offline after first load, so that lookups do not depend on anything but the phone. 43. As a learner, I want to install the page to my home screen, so that it opens like an app rather than a browser tab. 44. As a learner, I want the app to use my existing Jellyfin login, so that there is no second account or token to manage. 45. As a learner, I want to configure where Anki lives and which deck receives cards, so that Export goes somewhere I chose. 46. As a learner, I want to see whether Anki is currently reachable before I Export, so that a failure is predicted rather than discovered. 47. As a learner, I want the plugin to leave Jellyfin working normally when the app is idle, so that a study tool never degrades the media server for anyone else using it. ## Implementation Decisions Everything below is settled by the design session recorded in `CONTEXT.md` and ADRs 0001–0004. ### Shape The whole app is a Jellyfin plugin (ADR 0001). It serves the phone UI as a plugin web asset, same-origin, authenticated by the Jellyfin token the phone already holds. No sidecar service, no second auth system, no CORS. The phone side is an installable PWA, not a native shell. A Tauri wrapper was considered and deferred: it would re-introduce the seams the plugin decision removes (bundled assets, its own token) to buy protection against risks — chiefly IndexedDB eviction — that may never materialise. Wrapping a working PWA later is cheap. Target Jellyfin 10.11 (`dionysus` runs 10.11.11), not the 10.9 the official plugin template pins. Jellyfin runs in a container, so ffmpeg comes from the server's own media encoder and item paths are already container-relative. ### Division of labour C# handles sessions, media, storage, and export. The browser handles all Japanese. This split is deliberate — the mature deinflection code is TypeScript, and there is no reason to port it. ### Plugin modules - **CueTrack** — takes a raw subtitle track and returns normalized Cues: start, end, and text with styling, positioning, and ruby markup removed. Jellyfin types do not appear at its boundary. Text tracks only. - **SessionWatcher** — resolves which Playback Sessions belong to a user and exposes each one's Playhead, item, and stream indices. - **PlaybackRemote** — pause, unpause, and seek a Playback Session. - **MiningQueue** — persistence for Captures, in SQLite under the plugin data dir. Exposes add, list, drain, and the Known Snapshot. Plugin configuration is XML and is explicitly not used for this. - **MediaExtractor** — given an item and a Cue's span, produces a Clip and a Frame. The encoder is injected; the module's real logic is argument construction — padding, seek strategy, and where in the Cue the Frame is taken. - **AnkiExporter** — turns Captures into Notes. Owns note-type creation, media upload, duplicate checking, and partial-failure reporting. ### Phone modules - **Deinflector** — Surface Form to candidate Headwords with the reason chain. Lifted from 10ten Japanese Reader (GPL-3.0) along with its existing tests, rather than written fresh. - **DictionaryIndex** — the full `jmdict-eng` build bundled as a plugin asset, loaded once into IndexedDB, queried by longest match from a tap offset. No morphological tokenizer exists in this system (ADR 0002); lookup is tap-driven longest-match plus deinflection, which is why none is needed. - **PlayheadClock** — combines the last reported position, wall-clock elapsed time, and paused state into a current position, and from that selects the current Cue. - **SessionFeed** — subscribes to the server's session push and turns those messages into playback-state events. ### Contracts and behaviour - **Playhead cadence.** Clients report progress on a 10-second throttle while playing, but report immediately on pause, unpause, and any seek beyond a ~5-second drift. The phone therefore interpolates between reports and resyncs on every push; it must never assume a report is fresh. - **Auto-pause.** Tapping a word issues a pause to the Playback Session; dismissing issues an unpause. Both produce an immediate progress report, which resyncs the Playhead as a side effect. - **Capture contents.** Headword, Surface Form, the full Cue text, Cue start and end, item and media-source identity, the chosen dictionary entry as it read at capture time, and the deinflection reason chain. - **Note fields.** Headword, Reading, Surface Form, Cue, Definition, Clip, Frame, Source. A fixed note type is created on first Export — the Anki collection is starting empty, so there is nothing to map onto and no migration to consider. - **Known Snapshot.** Built from this app's own exported Notes only. The collection is fresh, so collection-wide indexing buys nothing yet; it is an additive change later if this is ever shared. - **Export transport.** Server-to-server HTTP to AnkiConnect on `dex-win` over the tailnet (ADR 0003). Because it is not a browser call, CORS never applies. - **Export is idempotent and resumable.** Successful Captures leave the queue; failed ones stay with their error, and a second run retries only those. ### Security AnkiConnect ships unauthenticated (`apiKey` defaults to none) and bound to loopback. Reaching it from the plugin requires binding it beyond loopback — bind specifically to the Tailscale interface address, never `0.0.0.0`, and set an `apiKey`. AnkiConnect can read and modify the entire collection; tailnet plus apiKey is acceptable, LAN or public is not. ### Licensing GPL-3.0, forced by the lifted deinflection table and consistent with Jellyfin's own GPL-3.0 plugin template (ADR 0004). JMdict's EDRDG licence requires visible attribution, which is a product requirement rendered in the lookup UI, not a README line. ## Testing Decisions A good test here asserts external behaviour at a module's boundary — text in, Cues out; Captures in, Anki calls out — and never reaches for internals. Tests should survive a rewrite of a module's insides. Fakes stand in for Jellyfin's encoder and for AnkiConnect; nothing in the suite requires a running Jellyfin, a running Anki, or a real media file beyond small fixtures. There is no prior art: this is a greenfield repo. Per standing practice, new logic gets tests regardless of repo precedent, so the suite starts with this work rather than being retrofitted. Tested — the six deep modules: - **CueTrack** — tag stripping, ruby handling, timing conversion, malformed and empty tracks, overlapping cues, and image-track rejection. - **MiningQueue** — add and drain round-trips, Known Snapshot contents, durability across restart, and behaviour when draining an empty queue. - **MediaExtractor** — argument construction against a fake encoder: padding applied to Cue boundaries, seek behaviour, Frame position within the Cue, and handling of a Cue that starts at zero or runs to the end of the file. - **AnkiExporter** — against a fake AnkiConnect: note-type creation on first run, media upload, duplicate skipping, partial failure leaving the right Captures queued, and a repeated run being a no-op. - **DictionaryIndex** — longest-match resolution from a tap offset, ranking of candidates, behaviour at the end of a Cue, and lookups that match nothing. - **PlayheadClock** — interpolation between reports, paused state freezing the clock, a seek arriving mid-interpolation, cue selection at exact boundaries, and a stale report that never gets superseded. **Deinflector** arrives with its own test suite from 10ten; those tests come across with the code rather than being rewritten. Not tested: SessionWatcher, PlaybackRemote, and SessionFeed. All three are thin pass-throughs to Jellyfin APIs where a test would assert the mock rather than any behaviour of ours. If server upgrades start breaking them — a live risk given the rebuild-per-major consequence in ADR 0001 — that judgement gets revisited. ## Out of Scope - **Image-based subtitles.** PGS and VOBSUB are bitmaps; supporting them means OCR. Shows with only image tracks get an explanatory message. - **Automatic highlighting of unknown words.** Rejected outright, and it is what makes the absence of a tokenizer possible (ADR 0002). Noticing your own gaps is the study activity. - **A known-words model.** The Known Snapshot is a duplicate check over our own Notes, not a model of what the learner knows. - **Frequency lists and word grading.** - **Reviewing the Mining Queue before Export.** Everything exports; Anki's browser is where mistakes get deleted. - **Editing Captures.** No Cue-boundary adjustment, no definition editing. - **Indexing the existing Anki collection.** Nothing to index yet. Additive later. - **A native Android app.** Deferred, with the migration path deliberately kept cheap. - **AnkiDroid direct-add**, and any export path that does not go through AnkiConnect. - **Generating .apkg files.** - **Multi-user support.** Personal tool; no per-user isolation beyond what Jellyfin already enforces on sessions. - **Languages other than Japanese**, and dictionaries other than JMdict. - **Publishing to a public Jellyfin plugin repository.** If that ever happens, the repo takes the `jellyfin-plugin-` prefix and the EDRDG commercial-use restriction needs a second look. ## Further Notes The riskiest assumption in the whole design is that a 10-second position report plus client-side interpolation feels tight enough to read along with. Everything else is conventional work; that one is a feel question no amount of planning settles. The cheapest possible slice — plugin skeleton, one live Cue rendering on the phone, no dictionary, no mining — tests it, and should come first. Open questions deliberately left for implementation, none of them architectural: how much padding a Clip gets and whether it may extend into an adjacent Cue; exactly where in a Cue the Frame is taken; how English tracks are chosen when a show has several; and how partial Export failures are surfaced in the UI. `dionysus` currently reports a Tailscale DNS health warning. Worth ruling out before debugging any connectivity problem between the plugin and `dex-win`.

aiko added the

ready-for-agent

label 2026-08-09 20:04:28 +00:00

aiko referenced this issue 2026-08-09 20:41:29 +00:00

Plugin builds against Jellyfin 10.11 and serves an authenticated page #2

aiko referenced this issue 2026-08-09 20:42:19 +00:00

Phone follows a live Playback Session and shows its Playhead #3

aiko referenced this issue 2026-08-09 20:42:20 +00:00

CueTrack normalizes subtitles; phone shows the transcript #4

aiko referenced this issue 2026-08-09 20:42:21 +00:00

PlayheadClock: transcript follows the show #5

aiko referenced this issue 2026-08-09 20:59:06 +00:00

Deploy the plugin to dionysus and verify on the Pixel #6

aiko added

ready-for-human

and removed

ready-for-agent

labels 2026-08-09 21:01:00 +00:00

aiko referenced this issue 2026-08-10 08:03:29 +00:00

Subscribe to the Sessions WebSocket instead of polling #7

aiko commented 2026-08-10 08:44:12 +00:00

Author

Owner

Copy link

This was generated by AI during triage.

Slice order, decided 2026-08-10. Lookup comes before mining, as two separate slices rather than the one the PRD implies.

Next slice — lookup only. Tap a word in the transcript, deinflect the Surface Form, look the Headword up in JMdict, show the entry. Nothing is saved. Brings in the full jmdict-eng bundled as a plugin asset, IndexedDB on the phone, the deinflection table lifted from 10ten (GPL-3.0, ADR 0004), and Yomichan-style longest-match from the tap point — no tokenizer (ADR 0002). Also the auto-pause on tap, since reading a definition while the show runs on is the problem it solves.

Slice after — mining. Capture, the Mining Queue in SQLite, the Known Snapshot duplicate badge, and Export to Anki with Clips and Frames (ADR 0003).

Why split: lookup is useful on its own, and it defers all the storage and export work behind the question of whether tap-to-lookup actually feels right on a phone. Same reasoning that made the first slice stop at 'does reading along work' — which paid off, since it did.

Nothing in the ADRs changes. This only reorders what gets built.

> *This was generated by AI during triage.* **Slice order, decided 2026-08-10.** Lookup comes before mining, as two separate slices rather than the one the PRD implies. **Next slice — lookup only.** Tap a word in the transcript, deinflect the Surface Form, look the Headword up in JMdict, show the entry. Nothing is saved. Brings in the full `jmdict-eng` bundled as a plugin asset, IndexedDB on the phone, the deinflection table lifted from 10ten (GPL-3.0, ADR 0004), and Yomichan-style longest-match from the tap point — no tokenizer (ADR 0002). Also the auto-pause on tap, since reading a definition while the show runs on is the problem it solves. **Slice after — mining.** Capture, the Mining Queue in SQLite, the Known Snapshot duplicate badge, and Export to Anki with Clips and Frames (ADR 0003). Why split: lookup is useful on its own, and it defers all the storage and export work behind the question of whether tap-to-lookup actually feels right on a phone. Same reasoning that made the first slice stop at 'does reading along work' — which paid off, since it did. Nothing in the ADRs changes. This only reorders what gets built.

aiko referenced this issue 2026-08-10 09:25:51 +00:00

DictionaryIndex: JMdict in SQLite, answering Candidate lookups #8

aiko referenced this issue 2026-08-10 09:26:17 +00:00

Tap a word in a Cue, see its Entry #9

aiko referenced this issue 2026-08-10 09:26:18 +00:00

One Cue at a time: the transcript becomes a carousel #10

aiko referenced this issue 2026-08-10 09:26:36 +00:00

Inflected words resolve: the Deinflector and Candidate ranking #11

aiko referenced this issue 2026-08-10 09:27:08 +00:00

The Lookup sheet: Entry carousel, painted Surface Form, and the three nothings #12

aiko referenced this issue 2026-08-10 09:27:08 +00:00

Deploy the lookup slice and tune it on the Pixel #13

aiko referenced this issue 2026-08-10 12:42:12 +00:00

PlaybackRemote: pause, unpause, and seek a Playback Session #14

aiko referenced this issue 2026-08-10 12:42:38 +00:00

Three regions and a footer: the layout the sheet can never overlap #15

aiko referenced this issue 2026-08-10 12:43:10 +00:00

Per-Cue transport: play, pause, and step a line at a time #16

aiko referenced this issue 2026-08-10 12:43:10 +00:00

Settings: a panel in the sheet, not a screen #17

aiko referenced this issue 2026-08-10 12:43:34 +00:00

Auto-pause, and letting the show follow the phone #18

aiko referenced this issue 2026-08-10 13:13:06 +00:00

Sign-in: port the approved mockup into the app #19

aiko referenced this issue 2026-08-10 14:04:38 +00:00

Do not offer transport a client cannot accept #20

aiko referenced this issue 2026-08-10 14:23:07 +00:00

Scrubbing back a Cue at a time actually works #21

aiko referenced this issue 2026-08-10 14:23:08 +00:00

The strip says frozen when nobody swiped #22

aiko referenced this issue 2026-08-10 14:23:08 +00:00

Put the Cue where the thumb is #23

aiko referenced this issue 2026-08-10 16:53:41 +00:00

Make the phone say what it thinks is happening #24

aiko referenced this issue 2026-08-11 21:01:25 +00:00

A report older than our own seek is not news #25

aiko referenced this issue 2026-08-11 21:41:45 +00:00

A swipe onto a panel the app was already heading for is still a swipe #26

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

No results found.

Labels

Clear labels   

needs-info

Waiting on reporter

  

needs-triage

Maintainer needs to evaluate

  

ready-for-agent

Fully specified, AFK-ready

  

ready-for-human

Needs human implementation

  

wontfix

Will not be actioned

No labels

needs-info

needs-triage

ready-for-agent

ready-for-human

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

mobrule/kurageyomi#1

No description provided.