89 lines
3.7 KiB
Markdown
89 lines
3.7 KiB
Markdown
# Lumi Companion transcription
|
|
|
|
`lumi_transcription` is a separately disableable Lumi plugin. It owns paired
|
|
devices, session state, model/runtime artifacts, worker supervision, caption
|
|
stabilization, delivery, diagnostics, tests, and WebUI routes. Core supplies the
|
|
shared Companion authentication and versioned WebSocket boundary.
|
|
|
|
## Runtime path
|
|
|
|
```text
|
|
selected OBS source -> same-user Companion IPC -> bounded audio queue
|
|
-> authenticated TLS WebSocket -> bounded Lumi session
|
|
-> supervised server-hosted whisper.cpp worker -> stabilized caption revisions
|
|
-> Companion -> OBS native caption boundary
|
|
```
|
|
|
|
Speech recognition never runs on the streaming computer. Raw audio is kept in
|
|
bounded memory and is not logged or written to disk. Lumi prefers the published
|
|
CUDA worker on an NVIDIA RTX 3060 or newer and retains the CPU worker as a
|
|
portable fallback.
|
|
|
|
## Install and pair
|
|
|
|
1. Enable **Lumi Transcription** under **Admin > Plugins**.
|
|
2. Use **Download Companion** from the Admin dashboard or Transcription page.
|
|
3. Extract the private ZIP and run `Lumi.Companion-Setup.exe` within 15 minutes.
|
|
4. Complete the per-user setup and install or repair the managed OBS integration
|
|
while OBS is closed.
|
|
5. Select the OBS microphone in Companion.
|
|
6. Run the short voice-free full-path check. A successful result remains valid
|
|
until a relevant source, integration, model, or host setting changes.
|
|
7. Use the dedicated transcription test when measuring real speech accuracy,
|
|
confidence, latency, and input level.
|
|
|
|
Production device HTTP and WebSocket traffic requires HTTPS/WSS. Plain
|
|
HTTP/WebSocket is allowed only for a device paired from the exact matching
|
|
loopback Lumi origin and only while both sides remain on loopback.
|
|
|
|
## Models and workers
|
|
|
|
The Admin page installs checksum-pinned model and worker artifacts only after
|
|
explicit confirmation. `small.en` is the recommended model;
|
|
`small.en-q5_1` and `base.en` remain available as lower-resource alternatives.
|
|
The worker is supervised, restarts are bounded, and an explicit
|
|
`LUMI_TRANSCRIPTION_WORKER` override remains available for nonstandard
|
|
installations.
|
|
|
|
The native worker source is under
|
|
`plugins/lumi_transcription/backend/transcription/worker-native`. Release
|
|
workers are built with `plugins/lumi_transcription/scripts/build-worker.ps1`.
|
|
|
|
## Operation and recovery
|
|
|
|
- Live sessions require an active OBS stream. Tests use simulated delivery and
|
|
never publish captions to Twitch.
|
|
- Disconnects pause delivery and retain the session/model for a bounded grace
|
|
period so a short reconnect can resume safely.
|
|
- Manual benchmark completion waits for the final confidence-bearing caption;
|
|
silence ends a test after ten seconds.
|
|
- Device revocation takes effect on the next authenticated request or
|
|
connection.
|
|
- Disabling the plugin unregisters routes, closes clients and sessions, stops
|
|
the worker, and leaves unrelated plugins running.
|
|
|
|
The Admin summary reports Companion, device, inference, and session health.
|
|
Detailed benchmark records remain inspectable for one hour and include
|
|
word/phrase latency and confidence views.
|
|
|
|
## Verification
|
|
|
|
```powershell
|
|
npm run verify:transcription
|
|
dotnet build companion/Lumi.Companion.sln -c Release -p:EnableWindowsTargeting=true
|
|
```
|
|
|
|
Target-machine acceptance can be recorded with
|
|
`companion/docs/performance-acceptance-template.md`.
|
|
|
|
## Remaining release limitation
|
|
|
|
The Windows installer is checksum-pinned but not yet code-signed, so Windows may
|
|
show an unknown-publisher warning. The release includes its licence, privacy
|
|
notice, third-party notices, and corresponding source for the GPL-licensed OBS
|
|
bridge.
|
|
|
|
See `docs/adr/0001-companion-transcription-boundaries.md`,
|
|
`protocol/companion-protocol-v1.md`, and
|
|
`companion/docs/obs-native-caption-compatibility-spike.md`.
|