103 lines
4.5 KiB
Markdown
103 lines
4.5 KiB
Markdown
# Lumi Companion transcription
|
|
|
|
`lumi_transcription` is a separately disableable Lumi plugin. It owns paired
|
|
devices, session state, model/runtime artifacts, worker supervision, caption
|
|
stabilization, delivery, diagnostics, tests, and WebUI routes. Core supplies the
|
|
shared Companion authentication and versioned WebSocket boundary.
|
|
|
|
## Runtime path
|
|
|
|
```text
|
|
selected OBS source -> same-user Companion IPC -> bounded audio queue
|
|
-> authenticated TLS WebSocket -> bounded Lumi session
|
|
-> supervised server-hosted whisper.cpp worker -> stabilized caption revisions
|
|
-> Companion -> OBS native caption boundary
|
|
```
|
|
|
|
Speech recognition never runs on the streaming computer. Raw audio is kept in
|
|
bounded memory and is not logged or written to disk. Lumi prefers the published
|
|
CUDA worker on an NVIDIA RTX 3060 or newer and retains the CPU worker as a
|
|
portable fallback.
|
|
|
|
## Install and pair
|
|
|
|
1. Enable **Lumi Transcription** under **Admin > Plugins**.
|
|
2. Use **Download Companion** from the Admin dashboard or Transcription page.
|
|
3. Extract the private ZIP and run `Lumi.Companion-Setup.exe` within 15 minutes.
|
|
4. Complete the per-user setup and install or repair the managed OBS integration
|
|
while OBS is closed.
|
|
5. Select the OBS microphone in Companion.
|
|
6. Run the short voice-free full-path check. A successful result remains valid
|
|
until a relevant source, integration, model, or host setting changes.
|
|
7. Use the dedicated transcription test when measuring real speech accuracy,
|
|
confidence, latency, and input level.
|
|
|
|
Production device HTTP and WebSocket traffic requires HTTPS/WSS. Plain
|
|
HTTP/WebSocket is allowed only for a device paired from the exact matching
|
|
loopback Lumi origin and only while both sides remain on loopback.
|
|
|
|
## Models and workers
|
|
|
|
The Admin page installs checksum-pinned model and worker artifacts only after
|
|
explicit confirmation. `small.en` is the recommended model;
|
|
`small.en-q5_1` and `base.en` remain available as lower-resource alternatives.
|
|
The worker is supervised, restarts are bounded, and an explicit
|
|
`LUMI_TRANSCRIPTION_WORKER` override remains available for nonstandard
|
|
installations.
|
|
|
|
The native worker source is under
|
|
`plugins/lumi_transcription/backend/transcription/worker-native`. Release
|
|
workers are built with `plugins/lumi_transcription/scripts/build-worker.ps1`.
|
|
|
|
## Operation and recovery
|
|
|
|
- Live sessions require an active OBS stream. Tests use simulated delivery and
|
|
never publish captions to Twitch.
|
|
- Disconnects pause delivery and retain the session/model for a bounded grace
|
|
period so a short reconnect can resume safely.
|
|
- Manual benchmark completion waits for the final confidence-bearing caption;
|
|
silence ends a test after ten seconds.
|
|
- Device revocation takes effect on the next authenticated request or
|
|
connection.
|
|
- Disabling the plugin unregisters routes, closes clients and sessions, stops
|
|
the worker, and leaves unrelated plugins running.
|
|
|
|
The Admin summary reports Companion, device, inference, and session health.
|
|
Detailed benchmark records remain inspectable for one hour and include
|
|
word/phrase latency and confidence views.
|
|
|
|
## OBS Bridge and Stream Testing
|
|
|
|
The same paired Companion identity and same-user OBS Bridge are also used for
|
|
private Stream Testing. Lumi issues the expiring RTMP/RTMPS destination from the
|
|
validated pairing hostname; plugins do not re-authenticate and Companion does
|
|
not invent or accept another host. Before changing OBS, Companion stores the
|
|
complete service snapshot in its DPAPI-protected recovery record. Every terminal
|
|
path uses that record until OBS confirms the exact service has been restored.
|
|
|
|
MediaMTX is installed, verified, configured, and supervised by Lumi Core on the
|
|
server. It is not shipped in Companion and does not run inference or transcode
|
|
media. See [`stream-testing.md`](stream-testing.md) for transport, firewall,
|
|
managed-runtime, playback, and recovery details.
|
|
|
|
## Verification
|
|
|
|
```powershell
|
|
npm run verify:transcription
|
|
dotnet build companion/Lumi.Companion.sln -c Release -p:EnableWindowsTargeting=true
|
|
```
|
|
|
|
Target-machine acceptance can be recorded with
|
|
`companion/docs/performance-acceptance-template.md`.
|
|
|
|
## Remaining release limitation
|
|
|
|
The Windows installer is checksum-pinned but not yet code-signed, so Windows may
|
|
show an unknown-publisher warning. The release includes its licence, privacy
|
|
notice, third-party notices, and corresponding source for the GPL-licensed OBS
|
|
bridge.
|
|
|
|
See `docs/adr/0001-companion-transcription-boundaries.md`,
|
|
`protocol/companion-protocol-v1.md`, and
|
|
`companion/docs/obs-native-caption-compatibility-spike.md`.
|