# Lumi Companion transcription (experimental)

## Delivered foundation

`lumi_transcription` is an independently disableable Lumi plugin. Core contains only a generic HTTP upgrade registry so plugins can attach versioned WebSocket transports. The plugin owns pairing/devices, session state, settings revisions, model/runtime manifests, worker supervision, caption stabilization, delivery, logs, routes, and shutdown.

The implemented transport path is:

```text
OBS bridge boundary -> same-user companion IPC -> bounded companion queue
-> authenticated TLS WebSocket -> bounded per-source Lumi buffer
-> supervised TranscriptionProvider -> stabilized revision-aware caption
-> WebSocket -> companion delivery adapter -> OBS native caption boundary
```

The repository contains a pinned native whisper.cpp rolling-window worker and a CPU build/protocol smoke test. It has not yet been benchmarked with a model on the target RTX 3060, packaged as the Windows CUDA runtime, or validated end to end. The repository also does not contain a complete signed OBS bridge installer. Accordingly, this branch does not claim live Twitch acceptance.

## Trust and privacy

- Pairing packages contain a cryptographically random credential that expires after 15 minutes and can be activated once. Lumi stores only its SHA-256 digest.
- Activation returns a revocable device secret once. The Windows companion stores it with current-user DPAPI.
- Device HTTP and WebSocket credentials require HTTPS/WSS. HTTP/WS is accepted only when an administrator downloads the package through an explicit loopback URL and Companion returns to that exact loopback origin, including its port. There is no environment-variable bypass.
- The OBS bridge never receives the Lumi credential and never connects to Lumi directly.
- Audio frames are capped at 200 ms and recovery buffers at five seconds. Old or excess frames are dropped; capture paths never block for inference.
- Raw audio is never logged or written to disk. JSON Lines diagnostics default to seven days and 256 MiB. Caption text can be disabled in diagnostics.

## Pairing and installation milestone

1. Serve Lumi through HTTPS and enable `lumi_transcription` under Admin > Plugins. Local development may use HTTP only from the exact `localhost` or loopback URL used to download Companion.
2. Use **Download Companion** in Admin or Plugins > Transcription. Lumi downloads and checksum-verifies the pinned Windows artifact, adds a short-lived single-use pairing file, and returns a private ZIP.
3. On the Windows streaming computer, extract the complete ZIP and start `Lumi.Companion.App.exe` within 15 minutes. Companion discovers the adjacent pairing file, exchanges it once, stores the credential with current-user DPAPI, and removes the pairing file after success.
4. Install a pinned runtime/model only after explicit confirmation. `small.en` is recommended; `small.en-q5_1` and `base.en` are fallbacks. Every artifact is checksum-verified before install.
5. Build the supervised worker with `plugins/lumi_transcription/scripts/build-worker.ps1`. Lumi discovers the installed CUDA worker first and the CPU worker second. `LUMI_TRANSCRIPTION_WORKER` remains an explicit override for nonstandard installations.

The Avalonia tray UI, self-contained paired ZIP, shared Lumi settings shell, Admin summary, and host-side model download/load controls now exist. The ZIP is not code-signed, so Windows may warn. The normal signed installer, managed bridge install/repair, live OBS source enumeration, and measured model benchmark wizard are not complete yet.

## Operation and recovery

Live start is rejected unless OBS reports an active stream. Test mode is allowed without streaming and sends captions only to the simulated delivery output. If a live stream or connection ends, delivery pauses and the session/model receive a 30-second grace period. Reconnection can resume that session; expiry finalizes and stops it. Recording without streaming does not create third-party delivery.

Device and capability revocation take effect on the next authenticated request/connection. Worker crashes are bounded to three restart attempts per minute. Disabling the plugin unregisters the WebSocket route, closes clients/sessions, stops the worker, clears timers, and leaves unrelated plugins operational.

## Diagnostics

Use the compact Lumi Companion section on Admin for connection, device, inference, session, and package health. Open the shared-shell Transcription settings page for provider, model, device, and log details. Recovery errors distinguish missing setup, unavailable inference, source inactivity, protocol incompatibility, and revoked access. The target-machine test must additionally record latency, resource, OBS missed-frame, audio-dropout, queue-drop, and network metrics using the acceptance template in `companion/docs/performance-acceptance-template.md`.

## Known limitations

- The managed .NET solution and self-contained Windows x64 application have been compiled in the development environment; the OBS SDK integration still requires its target toolchain and acceptance host.
- No real whisper.cpp streaming worker has been integrated or benchmarked.
- The bridge skeleton does not yet run its named-pipe worker or selected-source audio callback.
- Twitch toggleable caption behavior has not been tested; native API presence is not acceptance evidence.
- The downloadable bundle is an unsigned experimental self-contained ZIP, not the final signed installer.
- Bridge repair, source discovery/nested Program-scene evaluation, benchmark UX, and conflict-resolution UI remain pending.

See `docs/adr/0001-companion-transcription-boundaries.md`, `protocol/companion-protocol-v1.md`, and `companion/docs/obs-native-caption-compatibility-spike.md`.