# Lumi Companion transcription (experimental)

## Delivered foundation

`lumi_transcription` is an independently disableable Lumi plugin. Core contains only a generic HTTP upgrade registry so plugins can attach versioned WebSocket transports. The plugin owns pairing/devices, session state, settings revisions, model/runtime manifests, worker supervision, caption stabilization, delivery, logs, routes, and shutdown.

The implemented transport path is:

```text
OBS bridge boundary -> same-user companion IPC -> bounded companion queue
-> authenticated TLS WebSocket -> bounded per-source Lumi buffer
-> supervised TranscriptionProvider -> stabilized revision-aware caption
-> WebSocket -> companion delivery adapter -> OBS native caption boundary
```

The repository contains a pinned native whisper.cpp rolling-window worker and a CPU build/protocol smoke test. It has not yet been benchmarked with a model on the target RTX 3060, packaged as the Windows CUDA runtime, or validated end to end. The repository also does not contain a complete signed OBS bridge installer. Accordingly, this branch does not claim live Twitch acceptance.

## Trust and privacy

- Pairing packages contain a cryptographically random credential that expires after 15 minutes and can be activated once. Lumi stores only its SHA-256 digest.
- Activation returns a revocable device secret once. The Windows companion stores it with current-user DPAPI.
- Device HTTP and WebSocket credentials require HTTPS/WSS. Insecure transport is available only from localhost when `LUMI_COMPANION_DEV_ALLOW_INSECURE=1` is explicitly set.
- The OBS bridge never receives the Lumi credential and never connects to Lumi directly.
- Audio frames are capped at 200 ms and recovery buffers at five seconds. Old or excess frames are dropped; capture paths never block for inference.
- Raw audio is never logged or written to disk. JSON Lines diagnostics default to seven days and 256 MiB. Caption text can be disabled in diagnostics.

## Pairing and installation milestone

1. Serve Lumi through HTTPS and enable `lumi_transcription` under Admin > Plugins.
2. Open Plugins > Transcription and create a pairing package. It is a bootstrap package for the generic companion, not a credential that should be shared or committed.
3. On the Windows streaming computer, build `companion/Lumi.Companion.sln`, start `Lumi.Companion.App`, and choose the package from the Overview or Connection page.
4. Install a pinned runtime/model only after explicit confirmation. `small.en` is recommended; `small.en-q5_1` and `base.en` are fallbacks. Every artifact is checksum-verified before install.
5. Configure `LUMI_TRANSCRIPTION_WORKER` with the supervised streaming-worker executable once that worker is built for the target host.

The Avalonia tray UI and host-side model download/load controls now exist. The normal signed installer, managed bridge install/repair, live OBS source enumeration, and measured model benchmark wizard are not complete yet.

## Operation and recovery

Live start is rejected unless OBS reports an active stream. Test mode is allowed without streaming and sends captions only to the simulated delivery output. If a live stream or connection ends, delivery pauses and the session/model receive a 30-second grace period. Reconnection can resume that session; expiry finalizes and stops it. Recording without streaming does not create third-party delivery.

Device and capability revocation take effect on the next authenticated request/connection. Worker crashes are bounded to three restart attempts per minute. Disabling the plugin unregisters the WebSocket route, closes clients/sessions, stops the worker, clears timers, and leaves unrelated plugins operational.

## Diagnostics

Use the Transcription admin page for provider, model, device, and log health. Recovery errors distinguish missing setup, unavailable inference, source inactivity, protocol incompatibility, and revoked access. The target-machine test must additionally record latency, resource, OBS missed-frame, audio-dropout, queue-drop, and network metrics using the acceptance template in `companion/docs/performance-acceptance-template.md`.

## Known limitations

- The .NET SDK and OBS SDK are not available in the development environment used for this milestone, so those projects have not been compiled here.
- No real whisper.cpp streaming worker has been integrated or benchmarked.
- The bridge skeleton does not yet run its named-pipe worker or selected-source audio callback.
- Twitch toggleable caption behavior has not been tested; native API presence is not acceptance evidence.
- The companion shell is a protocol/bootstrap executable, not yet the Avalonia tray application.
- Installer signing, bridge repair, auto-start, source discovery/nested Program-scene evaluation, benchmark UX, and conflict-resolution UI remain pending.

See `docs/adr/0001-companion-transcription-boundaries.md`, `protocol/companion-protocol-v1.md`, and `companion/docs/obs-native-caption-compatibility-spike.md`.