Lumi/docs/lumi-companion-transcription.md

Lumi Companion transcription

lumi_transcription is a separately disableable Lumi plugin. It owns paired devices, session state, model/runtime artifacts, worker supervision, caption stabilization, delivery, diagnostics, tests, and WebUI routes. Core supplies the shared Companion authentication and versioned WebSocket boundary.

Runtime path

selected OBS source -> same-user Companion IPC -> bounded audio queue
-> authenticated TLS WebSocket -> bounded Lumi session
-> supervised server-hosted whisper.cpp worker -> stabilized caption revisions
-> Companion -> OBS native caption boundary

Speech recognition never runs on the streaming computer. Raw audio is kept in bounded memory and is not logged or written to disk. Lumi prefers the published CUDA worker on an NVIDIA RTX 3060 or newer and retains the CPU worker as a portable fallback.

Install and pair

1. Enable Lumi Transcription under Admin > Plugins.
2. Use Download Companion from the Admin dashboard or Transcription page.
3. Extract the private ZIP and run Lumi.Companion-Setup.exe within 15 minutes.
4. Complete the per-user setup and install or repair the managed OBS integration while OBS is closed.
5. Select the OBS microphone in Companion.
6. Run the short voice-free full-path check. A successful result remains valid until a relevant source, integration, model, or host setting changes.
7. Use the dedicated transcription test when measuring real speech accuracy, confidence, latency, and input level.

Production device HTTP and WebSocket traffic requires HTTPS/WSS. Plain HTTP/WebSocket is allowed only for a device paired from the exact matching loopback Lumi origin and only while both sides remain on loopback.

Models and workers

The Admin page installs checksum-pinned model and worker artifacts only after explicit confirmation. small.en is the recommended model; small.en-q5_1 and base.en remain available as lower-resource alternatives. The worker is supervised, restarts are bounded, and an explicit LUMI_TRANSCRIPTION_WORKER override remains available for nonstandard installations.

The native worker source is under plugins/lumi_transcription/backend/transcription/worker-native. Release workers are built with plugins/lumi_transcription/scripts/build-worker.ps1.

Operation and recovery

• Live sessions require an active OBS stream. Tests use simulated delivery and never publish captions to Twitch.
• Disconnects pause delivery and retain the session/model for a bounded grace period so a short reconnect can resume safely.
• Manual benchmark completion waits for the final confidence-bearing caption; silence ends a test after ten seconds.
• Device revocation takes effect on the next authenticated request or connection.
• Disabling the plugin unregisters routes, closes clients and sessions, stops the worker, and leaves unrelated plugins running.

The Admin summary reports Companion, device, inference, and session health. Detailed benchmark records remain inspectable for one hour and include word/phrase latency and confidence views.

Verification

npm run verify:transcription
dotnet build companion/Lumi.Companion.sln -c Release -p:EnableWindowsTargeting=true

Target-machine acceptance can be recorded with companion/docs/performance-acceptance-template.md.

Remaining release limitation

The Windows installer is checksum-pinned but not yet code-signed, so Windows may show an unknown-publisher warning. The release includes its licence, privacy notice, third-party notices, and corresponding source for the GPL-licensed OBS bridge.

See docs/adr/0001-companion-transcription-boundaries.md, protocol/companion-protocol-v1.md, and companion/docs/obs-native-caption-compatibility-spike.md.