Compare commits

..

14 Commits

Author SHA1 Message Date
Franz Rolfsvaag
5f05251155 release: publish Lumi 0.3.11 companion experience 2026-07-29 23:41:34 +02:00
Franz Rolfsvaag
0fb0089e1c release: publish Lumi 0.3.10 platform tenure 2026-07-27 09:42:58 +02:00
Franz Rolfsvaag
5c85ba400d release: publish Lumi 0.3.9 chat media and Auto VC controls 2026-07-27 00:10:01 +02:00
Franz Rolfsvaag
655547938a release: publish Lumi 0.3.8 native companion overlay 2026-07-26 22:06:34 +02:00
Franz Rolfsvaag
bc7bff0eb9 release: publish Lumi 0.3.7 2026-07-26 18:36:30 +02:00
Franz Rolfsvaag
480a34b0af fix: automate RTMPS certificates through DNS 2026-07-26 17:56:38 +02:00
Franz Rolfsvaag
dc87b7e8e2 release: publish Lumi 0.3.5 2026-07-26 17:38:59 +02:00
Franz Rolfsvaag
2d8dc86b20 feat: complete companion streaming and logging upgrades 2026-07-26 17:22:10 +02:00
Franz Rolfsvaag
2368c01eb0 fix: restore proxied Companion downloads 2026-07-26 14:22:51 +02:00
Franz Rolfsvaag
eb3d0baf43 Add Qwen skill docs, favicon, and ignore .QWEN.md 2026-07-25 21:58:39 +02:00
Franz Rolfsvaag
ba049d682b release: replace stream testing with MediaMTX 2026-07-25 20:57:22 +02:00
Franz Rolfsvaag
c321d2b1ff fix: prevent OBS stream test service crash 2026-07-25 00:04:03 +02:00
Franz Rolfsvaag
ea212c31b8 fix: surface stream test prerequisites 2026-07-24 23:39:14 +02:00
Franz Rolfsvaag
dceebce43a release: add private stream testing and media fixes 2026-07-24 22:12:39 +02:00
159 changed files with 13056 additions and 739 deletions

View File

@ -49,3 +49,9 @@ LUMI_OPERATOR_PRIVACY_URL=
# source checkout. Explicit overrides always win.
# LUMI_DEV_MODE=true
# LUMI_HOST=127.0.0.1
# Private OBS stream testing (optional; see docs/stream-testing.md).
# Public and local ingest ports are stored in Lumi settings and configured at
# Admin > Stream testing. Nginx Proxy Manager owns production TLS termination.
# Advanced/manual runtime override only; normal installations do not need this.
# LUMI_MEDIAMTX_PATH=/absolute/path/to/mediamtx

6
.gitignore vendored
View File

@ -31,4 +31,8 @@ Twitch.png
twitch-credentials-lumi.png
.secrets
.secrets-*
DEVNOTES.md
DEVNOTES.md
.QWEN.md
_*_changes.md
Lumi-current.zip
agent-task*.md

View File

@ -1,5 +1,91 @@
# Lumi changelog
## 0.3.11
- Released Companion 0.2.8 with click-to-capture global hotkeys, independent Windows Media Player, YouTube, Spotify, VLC, Apple Music, SoundCloud, TIDAL, Bandcamp, and Qobuz discovery, and an interactive edit mode built on the actual capture-excluded native overlay renderer.
- Refined Stats and Leaderboards to hide raw channel/server IDs, omit empty tenure boards, emphasize current tenure separately from recorded totals, and distinguish both values in comparisons.
- Added per-user favorite command tracking and favorite Expression Interaction summaries through an additive usage table; favorite command history begins with this release.
- Preserved existing Companion pairing, overlay and media settings, platform-tenure history, plugin data, OBS configuration, databases, uploads, models, and secrets through the normal updater.
## 0.3.10
- Added an extensible UTC interval-history engine for Twitch follows, subscriptions, moderators, editors, and VIPs; YouTube channel memberships and moderators; and Discord membership and server boosts.
- Added idempotent live-event handling, complete-snapshot reconciliation that never treats API failures as empty state, authoritative timestamp backfill for open intervals, and preserved history across restarts and account changes.
- Added contextual `current` and `total` age placeholders with deterministic years/months, selected-unit remainder flow, singular/plural output, and the shared embedded `{format.YY-MM-DD-HH-mm-ss}` grammar.
- Added Platform tenure summaries to Stats and scope-separated Platform tenure boards to Leaderboards, plus admin interval diagnostics with source, precision, and authoritative/first-observed provenance.
- Preserved all existing settings, identities, statistics, interval history, plugins, pairing records, databases, uploads, models, and secrets through additive migrations and the normal updater.
## 0.3.9
- Restored animated Twitch/BTTV and Discord emotes plus Discord GIF media across the existing normalized OBS and native Companion overlay paths, and rendered real Twitch badge artwork with safe fallbacks.
- Released Companion 0.2.7 with bounded multi-frame GIF/WebP decoding for the capture-excluded native overlay while retaining the existing OBS browser renderer and media workflow.
- Added configurable Auto VC room messages with the room-owner placeholder, persistent lock/unlock controls, timed ownership claiming, and short-lived action reports.
- Preserved all existing settings, lobbies, active rooms, ownership, permissions, pairing identities, Companion settings, overlays, databases, uploads, models, and secrets through additive migrations and the normal updater.
## 0.3.8
- Added the native Windows Lumi Overlay using the existing paired-device authentication, normalized provider streams, live-state tracking, and Companion plugin transport without exposing provider credentials.
- Added monitor-aware native layouts, visibility policies, capture-exclusion reporting, bounded animations, Twitch/BTTV emotes, local atomic settings, global hotkeys, preview controls, and direct tray integration.
- Preserved unsaved overlay configuration through live feed/status refreshes, corrected divided-layout percentage presentation, clarified current visibility versus button actions, improved labels and conditional controls, and kept unavailable monitor/source/event selections visible.
- Made single-page Song Overlay and Lumi Overlay plugins direct navigation entries while retaining collapsed grouping for genuinely multi-page plugins.
- Released Companion 0.2.6 through the existing verified installer/updater path while preserving pairing identity, Companion settings, server/plugin data, OBS configuration, and revoked-device state.
## 0.3.7
- Moved production Stream Testing TLS termination to Nginx Proxy Manager: Companion still receives RTMPS on the paired Lumi hostname while MediaMTX accepts only the decrypted local RTMP stream.
- Added DB-backed public and local ingest ports plus a full TLS and RTMP external-route check, while preserving direct loopback RTMP development and existing session authorization.
- Removed Lumi-owned ACME, DNS-provider, certificate-path, and challenge-route code without deleting any existing operator data.
## 0.3.6
- Added encrypted Domeneshop DNS-01 automation for production RTMPS certificates when OpenResty, Nginx Proxy Manager, or another HTTPS proxy owns the reserved HTTP challenge path.
- Added a Lumi-styled Stream Testing setup surface that verifies DNS credentials once, creates and removes only short-lived challenge records, and supports timed destructive credential removal.
- Kept HTTP-01 for installations that forward it correctly, improved proxy-specific failure guidance, and bounded authoritative DNS propagation within Companion's startup window.
- Released Companion 0.2.5 as the matching Transcription compatibility release while retaining OBS Bridge 0.2.5.
- Preserved all server, plugin, pairing, OBS, media, and existing certificate data; no external ACME package or certificate-path environment variable is required.
## 0.3.5
- Released Companion 0.2.4 with silent Lumi reconnect, a direct transcription on/off control, consistent collapsed plugin navigation, and a one-click route from an active Stream Test to its Lumi viewer.
- Made private-test captions progressively reveal speech instead of waiting for a full utterance, retained readable multi-line captions, and kept stale words moving out naturally.
- Added Lumi-managed RTMPS certificates for public production ingest while retaining HTTP and RTMP for explicit localhost/private development use.
- Refreshed Song Overlay capture and delivery, shared Companion authentication across installed plugins, and completed durable redacted logging across core and bundled plugins.
- Released OBS Bridge 0.2.5 and synchronized the updated Transcription and Song Overlay plugins without replacing preserved server, plugin, pairing, or OBS data.
## 0.3.4
- Fixed production Companion pairing and download generation behind Lumi's private HTTPS reverse proxy, while continuing to reject forged forwarding headers from public clients.
- Reused the same guarded request-origin policy for Companion HTTP, pairing, settings, and WebSocket traffic, with explicit regression coverage for private proxy HTTPS, public spoofing, and malformed hosts.
- Standardized operational failures on Lumi's durable redacting logger, removed a placeholder metrics API that silently discarded events, and documented the actual logging and feature-owned metrics boundaries.
- Included the existing Lumi favicon and Qwen workspace housekeeping from the branch without changing preserved server or plugin data.
## 0.3.3
- Isolated private tests in a dedicated libobs output, bypassed account OAuth stream-key mutation, retained crash-safe exact service restoration, and corrected OBS encoder ownership so failed outputs cannot corrupt a later retry.
- Replaced the Stream Testing FFmpeg listener and rendition transcoder with a checksum-pinned, Lumi-managed MediaMTX runtime that receives OBS RTMP/RTMPS and remuxes the genuine source representation to low-latency HLS without recording.
- Added explicit MediaMTX install, repair, and receiver-health actions; loopback-only API, metrics, and HLS; exact per-session hashed publisher authorization; bounded supervision and diagnostics; and authenticated same-origin streaming playback.
- Kept OBS on the existing paired Lumi hostname, made unencrypted RTMP private-network-only by default, removed the extra FFmpeg/package requirement, and retained crash-safe exact OBS destination restoration and private caption delivery.
- Updated Stream Testing to show only real source quality, factual OBS/MediaMTX/player diagnostics, and receiver warnings while preserving overlay, update, Companion plugin, transcription, and Song Overlay behavior.
- Released Companion 0.2.3 with OBS Bridge 0.2.4 so the new private-output and recovery behavior reaches installed streaming computers through the normal updater.
## 0.3.2
- Fixed an OBS crash at private Stream Test startup by respecting the OBS Frontend API's borrowed streaming-service reference instead of releasing OBS-owned memory.
- Rebuilt the managed OBS Bridge with a new version so Companion clearly requires and installs the corrected integration before another Stream Test can start.
## 0.3.1
- Fixed apparently inert Stream Testing starts by surfacing the server rejection immediately in the Stream Testing panel and local diagnostic log.
- Added bounded Windows FFmpeg discovery for verified WinGet and conventional installations, while preserving explicit `LUMI_FFMPEG_PATH` configuration and Linux PATH behavior.
- Added explicit loaded/installed/bundled OBS Bridge version comparison, update-required notices and actions on Overview and Connection & device, and blocked Stream Testing until an outdated bridge is updated and loaded.
## 0.3.0
- Added admin-only private stream testing for the real OBS output with expiring authenticated sessions, supervised FFmpeg ingest, source/720p/480p no-upscale HLS, automatic/manual quality, audio, fullscreen, reused captions, real OBS/receiver diagnostics, bounded cleanup, and a deterministic test pattern.
- Added a crash-safe two-phase OBS destination handoff: Companion protects the complete prior service with current-user DPAPI before redirecting, then restores it on stop, expiry, receiver failure, WebSocket loss, Companion exit, OBS restart, or startup recovery.
- Made public video overlays pixel-only and playback-lifecycle aware, and converted audio sources from visual canvas objects into invisible managed outputs with inspector playback controls and ignored legacy layout data.
- Made Companion update checks safely repeatable after success, no-update, failure, cancellation, or rapid repeated clicks.
## 0.2.27
- Completed bundled-plugin synchronization automatically on the first startup after a legacy core-only update, allowing production hosts on 0.2.25 to receive the Companion transcription and Song Overlay plugins in the same update flow.

View File

@ -55,10 +55,12 @@ You can set these in `.env` or change role IDs in **Admin → Settings**.
Use **Admin → Plugins** to install, enable, update, or uninstall plugins.
You can also create a local plugin from the WebUI.
The experimental `experimental-companion` branch includes the independent Lumi
Companion transcription foundation. Its current scope, trust boundaries, setup,
and unverified target-machine work are documented in
Lumi Companion provides the streaming-computer boundary for transcription,
media capture, and private stream testing. Its trust boundaries and setup are documented in
[`docs/lumi-companion-transcription.md`](docs/lumi-companion-transcription.md).
Private OBS output testing, Lumi-managed MediaMTX setup, recovery behavior, and
network safety are documented in
[`docs/stream-testing.md`](docs/stream-testing.md).
## Updates and recovery

View File

@ -21,6 +21,12 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Lumi.Companion.Transcriptio
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Lumi.Companion.SongOverlay", "plugins\Lumi.Companion.SongOverlay\Lumi.Companion.SongOverlay.csproj", "{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "tests", "tests", "{0AB3BF05-4346-4AA6-1389-037BE0695223}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Lumi.Companion.Core.Tests", "tests\Lumi.Companion.Core.Tests\Lumi.Companion.Core.Tests.csproj", "{AAFDF982-BE30-405F-A39B-E82F1B11D46A}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Lumi.Companion.Overlay", "plugins\Lumi.Companion.Overlay\Lumi.Companion.Overlay.csproj", "{E6AC503B-9723-441C-8882-D0EC4A845F9D}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
@ -43,18 +49,6 @@ Global
{0B3C2C62-2CCB-43C9-9AF9-769AC7F22285}.Release|x64.Build.0 = Release|Any CPU
{0B3C2C62-2CCB-43C9-9AF9-769AC7F22285}.Release|x86.ActiveCfg = Release|Any CPU
{0B3C2C62-2CCB-43C9-9AF9-769AC7F22285}.Release|x86.Build.0 = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|x64.ActiveCfg = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|x64.Build.0 = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|x86.ActiveCfg = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|x86.Build.0 = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|Any CPU.Build.0 = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|x64.ActiveCfg = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|x64.Build.0 = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|x86.ActiveCfg = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|x86.Build.0 = Release|Any CPU
{3703AF57-828D-4E2F-BFBA-A95833555A5A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{3703AF57-828D-4E2F-BFBA-A95833555A5A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{3703AF57-828D-4E2F-BFBA-A95833555A5A}.Debug|x64.ActiveCfg = Debug|Any CPU
@ -115,6 +109,42 @@ Global
{49C19CA8-4669-4C33-B3A8-9CB934CD1171}.Release|x64.Build.0 = Release|Any CPU
{49C19CA8-4669-4C33-B3A8-9CB934CD1171}.Release|x86.ActiveCfg = Release|Any CPU
{49C19CA8-4669-4C33-B3A8-9CB934CD1171}.Release|x86.Build.0 = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|x64.ActiveCfg = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|x64.Build.0 = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|x86.ActiveCfg = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Debug|x86.Build.0 = Debug|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|Any CPU.Build.0 = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|x64.ActiveCfg = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|x64.Build.0 = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|x86.ActiveCfg = Release|Any CPU
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2}.Release|x86.Build.0 = Release|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Debug|x64.ActiveCfg = Debug|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Debug|x64.Build.0 = Debug|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Debug|x86.ActiveCfg = Debug|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Debug|x86.Build.0 = Debug|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Release|Any CPU.ActiveCfg = Release|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Release|Any CPU.Build.0 = Release|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Release|x64.ActiveCfg = Release|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Release|x64.Build.0 = Release|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Release|x86.ActiveCfg = Release|Any CPU
{AAFDF982-BE30-405F-A39B-E82F1B11D46A}.Release|x86.Build.0 = Release|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Debug|Any CPU.Build.0 = Debug|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Debug|x64.ActiveCfg = Debug|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Debug|x64.Build.0 = Debug|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Debug|x86.ActiveCfg = Debug|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Debug|x86.Build.0 = Debug|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Release|Any CPU.ActiveCfg = Release|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Release|Any CPU.Build.0 = Release|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Release|x64.ActiveCfg = Release|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Release|x64.Build.0 = Release|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Release|x86.ActiveCfg = Release|Any CPU
{E6AC503B-9723-441C-8882-D0EC4A845F9D}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
@ -127,5 +157,7 @@ Global
{2F0041FD-FD06-4312-BF1E-180F0201EB97} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B}
{49C19CA8-4669-4C33-B3A8-9CB934CD1171} = {07D57EEB-2F50-60C4-C011-FE4FA775C9A8}
{F72BAFAD-17D3-4EF2-8690-B3B5646556C2} = {07D57EEB-2F50-60C4-C011-FE4FA775C9A8}
{AAFDF982-BE30-405F-A39B-E82F1B11D46A} = {0AB3BF05-4346-4AA6-1389-037BE0695223}
{E6AC503B-9723-441C-8882-D0EC4A845F9D} = {07D57EEB-2F50-60C4-C011-FE4FA775C9A8}
EndGlobalSection
EndGlobal

View File

@ -2,9 +2,9 @@
Lumi Companion is the Windows streaming-computer client for Lumi. It provides a
single paired-device shell for Companion plugins, including server-hosted
transcription and Song Overlay capture. The app is single-instance, installs per
Windows user, closes to the notification area, and preserves pairing and settings
across updates.
transcription, Song Overlay capture, and the native Lumi monitor overlay. The app
is single-instance, installs per Windows user, closes to the notification area,
and preserves pairing and settings across updates.
## Install and pair
@ -25,6 +25,21 @@ DPAPI-protected device identity, so reinstalling does not create a duplicate
paired device. Pairing packages expire after 15 minutes, work once, and must not
be shared or committed.
## Companion features
Global hotkey fields are buttons: select one, press the desired modifier/key
combination, then release it to save the draft. Capture cancels after seven
seconds without a complete shortcut.
Song Overlay discovers independently enabled Windows media sessions. Windows
Media Player, YouTube/YouTube Music, and Spotify are enabled by default; VLC,
iTunes/Apple Music, SoundCloud, TIDAL, Bandcamp, and Qobuz can be enabled
individually.
Lumi Overlay Edit Mode opens the real monitor renderer with sample content and
interactive container outlines. Drag a container to move it or its lower-right
corner to resize it; the app retains these as an unsaved draft until Save.
## Security boundaries
- Companion plugins inherit the shell's paired-device authentication. Plugins do
@ -48,10 +63,49 @@ recording. Applying an update replaces the installed executable, bundled
components, and legal bundle, then restarts Companion. Pairing credentials,
settings, and plugin state remain in the per-user data directory.
If the paired Lumi host is temporarily unavailable, Companion remains paired and
reconnects quietly in the background with bounded backoff. The Overview and
Connection pages show the offline state without repeated dialogs; **Retry now**
remains available for an immediate manual attempt.
Localhost development builds can also receive checksum-addressed same-version
updates without publishing a release. See
[Localhost development updates](../docs/local-development-updates.md).
## Transcription control
The Transcription page has a persistent **Generate and include captions** switch,
also available from the tray menu. Turning it off stops the active transcription
session, excludes captions from live and private-test output, and stops sending
microphone audio to Lumi for speech recognition. Private Stream Testing remains
available as a video-only test. Turning the switch back on restores automatic
caption startup with OBS and can start captions while a private test is already
running.
## Private Stream Testing
The core **Stream Testing** page asks the paired Lumi host for an expiring
destination, then uses the managed OBS Bridge to snapshot the complete current
OBS streaming service before installing that destination. OBS sends its ordinary
encoded video and audio to the existing paired Lumi hostname on the configured
RTMP/RTMPS port; Companion does not need a second login, destination editor, or
local media runtime.
While a private test is active, Companion exposes **Open stream viewer**, which
opens the paired Lumi host directly at **Admin > Stream testing**.
Lumi owns and verifies the MediaMTX receiver. MediaMTX remuxes the source stream
for same-origin Admin playback without transcoding or recording it. In v0.3.3,
`Source` is therefore the normal and only quality choice unless a future
provider supplies genuine additional representations.
Stopping the test, losing the Lumi connection, receiver failure, Companion
exit, expiry, or an OBS restart all enter the same recovery path. Companion
keeps the DPAPI-protected recovery record until the OBS Bridge confirms that the
exact saved service type and settings were restored and saved. If automatic
restoration cannot finish while OBS is unavailable, the recovery remains
visible and can be retried after OBS and the bridge reconnect.
## Build and verify
Requirements:
@ -83,6 +137,16 @@ Run Song Overlay's focused verification with:
companion/scripts/verify-song-overlay.ps1
```
Run the native Lumi Overlay feed, layout/lifecycle, settings, and full managed
build verification with:
```powershell
companion/scripts/verify-lumi-overlay.ps1
```
Native overlay behavior and capture-exclusion limitations are documented in
[`docs/companion-native-overlay.md`](../docs/companion-native-overlay.md).
Generated installers, pairing packages, credentials, build output, and logs must
not be committed.

View File

@ -1,6 +1,6 @@
# Song Overlay Companion integration
This integration adds a provider-neutral media event source to Lumi Companion. Spotify is the only available provider in this release, but the Companion protocol and runtime isolate provider-specific behavior behind `IMediaProvider`.
This integration adds provider-neutral Windows media-session discovery to Lumi Companion. It can identify Spotify, Windows Media Player, YouTube and YouTube Music in Chrome or Edge, VLC, iTunes and Apple Music, SoundCloud, TIDAL, Bandcamp, and Qobuz. Each source is independently switchable; Windows Media Player, YouTube, and Spotify are enabled by default.
## Navigation contract
@ -13,8 +13,8 @@ Plugin actions are caught at the shell boundary so a routine plugin exception do
## Song Overlay flow
1. Windows Global System Media Transport Controls exposes Spotify playback state.
2. The provider raises media, playback, timeline, and session-availability events.
1. Windows Global System Media Transport Controls exposes playback from enabled desktop and browser sources.
2. The provider identifies the source where Windows exposes enough app or metadata context, then raises media, playback, timeline, and session-availability events.
3. The runtime sends only meaningful deltas to Lumi: track change, inferred next/previous, play, resume, pause, stop, seek, metadata enrichment, or a sparse recovery heartbeat.
4. Playback progress is projected from the last event. Continuous progress messages are not sent.
5. Cover art is resized and sent only with track metadata.

View File

@ -1,5 +1,5 @@
#ifndef AppVersion
#define AppVersion "0.1.0"
#define AppVersion "0.2.8"
#endif
#ifndef SourceRoot
#error SourceRoot must point at the self-contained Companion publish directory.

View File

@ -1,5 +1,5 @@
LUMI COMPANION PRIVACY NOTICE
Effective date: 23 July 2026
Effective date: 25 July 2026
This notice describes the default data behaviour of the Lumi Companion Windows application. It does not replace the privacy notice of the person or organisation operating the Lumi host to which Companion is paired.
@ -35,11 +35,14 @@ Depending on the features you enable, Companion may send:
- the computer or device name, installation identifier, Companion version, protocol version, capabilities, and component status;
- OBS state and selected-source information;
- audio from the OBS source you select, converted to 16 kHz mono signed 16-bit PCM and transmitted in bounded frames;
- when you explicitly start a private Stream Test, the real OBS audio/video output, temporary Stream Test destination facts, and output diagnostics required to preview and troubleshoot that output;
- readiness, test, benchmark, connection, and diagnostic information; and
- settings or control messages required for enabled Companion features.
Raw audio is held in bounded memory and is not written to disk by Companion by default. Audio is transmitted to the paired Lumi host for server-side processing. The Host Operator controls the server-side speech model, storage, logs, caption delivery, integrations, and retention. Review the Host Operators notice before enabling capture.
Private Stream Test media is sent only after you start a test. The default Lumi host repackages it into a short in-memory live window for authenticated administrators and does not retain a recording. A modified or independently operated host may behave differently; review the Host Operators notice and configuration.
You are responsible for having authority to capture and transmit audio or other information, including informing participants and obtaining consent where required.
4. NETWORK CONNECTIONS

View File

@ -2,8 +2,8 @@
# Copyright (c) 2026 OokamiKunTV
cmake_minimum_required(VERSION 3.28)
project(lumi-obs-bridge VERSION 0.1.0 LANGUAGES CXX)
set(LUMI_BRIDGE_VERSION "0.1.0-development" CACHE STRING "Lumi Companion bridge release version")
project(lumi-obs-bridge VERSION 0.2.5 LANGUAGES CXX)
set(LUMI_BRIDGE_VERSION "0.2.5-development" CACHE STRING "Lumi Companion bridge release version")
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

View File

@ -3,6 +3,7 @@
#include <obs-module.h>
#include <obs-frontend-api.h>
#include <util/config-file.h>
#include <nlohmann/json.hpp>
#include <windows.h>
#include <bcrypt.h>
@ -67,6 +68,9 @@ static std::atomic_bool obs_state_dirty{true};
static std::atomic_uint32_t audio_sequence{0};
static obs_source_t *captured_source = nullptr;
static std::mutex captured_source_mutex;
static std::mutex stream_test_mutex;
static std::string stream_test_session;
static obs_output_t *stream_test_output = nullptr;
static std::string wide_to_utf8(const std::wstring &value)
{
@ -324,15 +328,265 @@ static json source_list_message()
static json obs_state_message()
{
obs_video_info video{};
obs_get_video_info(&video);
return {{"type", "obs_state"}, {"protocol_version", protocol_version}, {"version", obs_get_version_string()},
{"streaming", obs_frontend_streaming_active()}, {"recording", obs_frontend_recording_active()}};
{"streaming", obs_frontend_streaming_active()}, {"recording", obs_frontend_recording_active()},
{"output_width", video.output_width}, {"output_height", video.output_height},
{"fps_num", video.fps_num}, {"fps_den", video.fps_den}};
}
struct service_command_context {
bool begin = false;
std::string server;
std::string key;
std::string session_id;
std::string restore_type;
std::string restore_settings;
bool ok = false;
std::string error;
std::string snapshot_type;
std::string snapshot_settings;
};
static bool restore_frontend_service(const std::string &type, const std::string &settings_json, std::string &error)
{
obs_data_t *settings = obs_data_create_from_json(settings_json.c_str());
if (!settings) {
error = "The encrypted OBS recovery snapshot is invalid.";
return false;
}
obs_service_t *restored = obs_service_create(type.c_str(), "Restored streaming service", settings, nullptr);
obs_data_release(settings);
if (!restored) {
error = "OBS could not restore the saved streaming service.";
return false;
}
obs_frontend_set_streaming_service(restored);
obs_frontend_save_streaming_service();
obs_service_release(restored);
return true;
}
static obs_output_t *create_stream_test_output(obs_service_t *service, std::string &error)
{
config_t *profile = obs_frontend_get_profile_config();
const char *mode = profile ? config_get_string(profile, "Output", "Mode") : nullptr;
const bool advanced = mode && std::strcmp(mode, "Advanced") == 0;
const char *configured_video = profile
? config_get_string(profile, advanced ? "AdvOut" : "SimpleOutput", advanced ? "Encoder" : "StreamEncoder")
: nullptr;
const char *configured_audio = profile && advanced
? config_get_string(profile, "AdvOut", "AudioEncoder")
: "ffmpeg_aac";
const char *video_id = configured_video && *configured_video ? configured_video : "obs_x264";
const char *audio_id = configured_audio && *configured_audio ? configured_audio : "ffmpeg_aac";
obs_data_t *video_settings = obs_encoder_defaults(video_id);
if (!video_settings) video_settings = obs_data_create();
char *profile_path = obs_frontend_get_current_profile_path();
if (profile_path) {
const std::string encoder_path = std::string(profile_path) + "/streamEncoder.json";
obs_data_t *saved = obs_data_create_from_json_file(encoder_path.c_str());
if (saved) {
obs_data_apply(video_settings, saved);
obs_data_release(saved);
}
bfree(profile_path);
}
if (!obs_data_has_user_value(video_settings, "bitrate"))
obs_data_set_int(video_settings, "bitrate", profile ? config_get_uint(profile, "SimpleOutput", "VBitrate") : 6000);
obs_data_set_int(video_settings, "keyint_sec", 2);
obs_encoder_t *video = obs_video_encoder_create(video_id, "Lumi private stream video", video_settings, nullptr);
obs_data_release(video_settings);
if (!video && std::strcmp(video_id, "obs_x264") != 0) {
obs_data_t *fallback = obs_encoder_defaults("obs_x264");
video = obs_video_encoder_create("obs_x264", "Lumi private stream video", fallback, nullptr);
if (fallback) obs_data_release(fallback);
}
if (!video) {
error = "OBS could not create a video encoder for the private output.";
return nullptr;
}
obs_encoder_set_video(video, obs_get_video());
if (advanced && profile && config_get_bool(profile, "AdvOut", "Rescale")) {
const char *resolution = config_get_string(profile, "AdvOut", "RescaleRes");
unsigned int width = 0, height = 0;
if (resolution && std::sscanf(resolution, "%ux%u", &width, &height) == 2 && width && height)
obs_encoder_set_scaled_size(video, width, height);
}
obs_data_t *audio_settings = obs_encoder_defaults(audio_id);
if (!audio_settings) audio_settings = obs_data_create();
const uint64_t audio_bitrate = profile
? config_get_uint(profile, advanced ? "AdvOut" : "SimpleOutput", advanced ? "Track1Bitrate" : "ABitrate")
: 160;
obs_data_set_int(audio_settings, "bitrate", audio_bitrate ? static_cast<long long>(audio_bitrate) : 160);
obs_encoder_t *audio = obs_audio_encoder_create(audio_id, "Lumi private stream audio", audio_settings, 0, nullptr);
obs_data_release(audio_settings);
if (!audio && std::strcmp(audio_id, "ffmpeg_aac") != 0) {
obs_data_t *fallback = obs_encoder_defaults("ffmpeg_aac");
audio = obs_audio_encoder_create("ffmpeg_aac", "Lumi private stream audio", fallback, 0, nullptr);
if (fallback) obs_data_release(fallback);
}
if (!audio) {
obs_encoder_release(video);
error = "OBS could not create an audio encoder for the private output.";
return nullptr;
}
obs_encoder_set_audio(audio, obs_get_audio());
obs_data_t *output_settings = obs_output_defaults("rtmp_output");
if (!output_settings) output_settings = obs_data_create();
obs_data_set_string(output_settings, "bind_ip", "default");
obs_data_set_bool(output_settings, "dyn_bitrate", false);
obs_output_t *output = obs_output_create("rtmp_output", "Lumi private stream output", output_settings, nullptr);
obs_data_release(output_settings);
if (!output) {
obs_encoder_release(audio);
obs_encoder_release(video);
error = "OBS could not create the private RTMP output.";
return nullptr;
}
obs_output_set_video_encoder(output, video);
obs_output_set_audio_encoder(output, audio, 0);
obs_encoder_release(audio);
obs_encoder_release(video);
obs_output_set_service(output, service);
obs_output_set_reconnect_settings(output, 0, 1);
return output;
}
static void apply_stream_test_service(void *opaque)
{
auto &context = *static_cast<service_command_context *>(opaque);
if (context.begin) {
if (obs_frontend_streaming_active()) {
context.error = "OBS is already streaming. Stop the current output before starting a private test.";
return;
}
obs_service_t *existing = obs_frontend_get_streaming_service();
if (!existing) {
context.error = "OBS does not have a streaming service to restore after the test.";
return;
}
const char *type = obs_service_get_id(existing);
obs_data_t *settings = obs_service_get_settings(existing);
context.snapshot_type = type ? type : "";
context.snapshot_settings = settings ? obs_data_get_json(settings) : "";
if (settings) obs_data_release(settings);
// obs_frontend_get_streaming_service returns a borrowed reference.
// Releasing it here destroys OBS's current service; SetService then
// releases the same object again while replacing it.
if (context.snapshot_type.empty() || context.snapshot_settings.empty()) {
context.error = "The current OBS stream service could not be snapshotted safely.";
return;
}
obs_data_t *test_settings = obs_data_create();
obs_data_set_string(test_settings, "server", context.server.c_str());
obs_data_set_string(test_settings, "key", context.key.c_str());
obs_data_set_bool(test_settings, "use_auth", false);
obs_service_t *test = obs_service_create("rtmp_custom", "Lumi private stream test", test_settings, nullptr);
obs_data_release(test_settings);
if (!test) {
context.error = "OBS could not create the private test service.";
return;
}
// Use a dedicated libobs output so platform OAuth cannot replace the
// expiring private stream key. Point the frontend at the same private
// service only as a fail-safe: pressing OBS's normal Start button
// cannot accidentally publish to the saved public destination.
obs_frontend_set_streaming_service(test);
obs_output_t *output = create_stream_test_output(test, context.error);
if (!output) {
restore_frontend_service(context.snapshot_type, context.snapshot_settings, context.error);
obs_service_release(test);
return;
}
if (!obs_output_start(output)) {
const char *error = obs_output_get_last_error(output);
const std::string start_error = error && *error ? error : "OBS could not start the private output.";
obs_output_release(output);
restore_frontend_service(context.snapshot_type, context.snapshot_settings, context.error);
obs_service_release(test);
context.error = start_error;
return;
}
obs_service_release(test);
{
std::scoped_lock lock(stream_test_mutex);
if (stream_test_output) obs_output_release(stream_test_output);
stream_test_output = output;
stream_test_session = context.session_id;
}
context.ok = true;
return;
}
obs_output_t *output = nullptr;
{
std::scoped_lock lock(stream_test_mutex);
output = stream_test_output;
stream_test_output = nullptr;
stream_test_session.clear();
}
if (output) {
if (obs_output_active(output)) obs_output_force_stop(output);
if (obs_output_active(output)) {
std::scoped_lock lock(stream_test_mutex);
stream_test_output = output;
context.error = "The private OBS output is still stopping. Try restoration again.";
return;
}
}
if (output) obs_output_release(output);
context.ok = restore_frontend_service(context.restore_type, context.restore_settings, context.error);
}
static json stream_test_metrics_message()
{
std::string session_id;
obs_output_t *output = nullptr;
{
std::scoped_lock lock(stream_test_mutex);
session_id = stream_test_session;
if (stream_test_output) output = obs_output_get_ref(stream_test_output);
}
if (session_id.empty()) return json();
obs_video_info video{};
obs_get_video_info(&video);
json value{{"type", "stream_test_metrics"}, {"protocol_version", protocol_version}, {"session_id", session_id},
{"active", output && obs_output_active(output)}, {"width", video.output_width}, {"height", video.output_height},
{"fps", video.fps_den ? static_cast<double>(video.fps_num) / video.fps_den : 0.0}};
if (output) {
value["total_bytes"] = obs_output_get_total_bytes(output);
value["dropped_frames"] = obs_output_get_frames_dropped(output);
value["total_frames"] = obs_output_get_total_frames(output);
value["congestion"] = obs_output_get_congestion(output);
obs_encoder_t *encoder = obs_output_get_video_encoder(output);
value["encoder"] = encoder && obs_encoder_get_id(encoder) ? obs_encoder_get_id(encoder) : "";
// obs_output_get_video_encoder returns a borrowed reference.
obs_output_release(output);
}
return value;
}
static bool output_caption(const std::string &text, double display_seconds)
{
if (!obs_frontend_streaming_active() || text.empty()) return false;
obs_output_t *output = obs_frontend_get_streaming_output();
if (text.empty()) return false;
obs_output_t *output = nullptr;
{
std::scoped_lock lock(stream_test_mutex);
if (stream_test_output) output = obs_output_get_ref(stream_test_output);
}
if (!output) output = obs_frontend_get_streaming_output();
if (!output) return false;
if (!obs_output_active(output)) {
obs_output_release(output);
return false;
}
obs_output_output_caption_text2(output, text.c_str(), std::clamp(display_seconds, 0.5, 10.0));
obs_output_release(output);
return true;
@ -344,8 +598,7 @@ static std::optional<json> handle_command(const json &message)
if (type == "select_sources") {
const auto uuid = message.value("primary_source_uuid", "");
const auto installed = select_source(uuid);
source_list_dirty.store(true, std::memory_order_release);
blog(installed ? LOG_INFO : LOG_WARNING, "[Lumi Companion] %s selected OBS audio source %s",
blog(installed ? LOG_INFO : LOG_WARNING, "[Lumi Companion] event=audio_source_selection_changed %s selected OBS audio source %s",
installed ? "Attached" : "Could not attach", uuid.c_str());
return json{{"type", "selection_state"}, {"protocol_version", protocol_version}, {"source_uuid", uuid}, {"attached", installed}};
} else if (type == "caption" && message.contains("payload")) {
@ -353,6 +606,62 @@ static std::optional<json> handle_command(const json &message)
const auto text = payload.value("stable_text", "");
const auto duration = payload.value("display_seconds", 2.0);
output_caption(text, duration);
} else if (type == "stream_test_snapshot") {
service_command_context context;
obs_queue_task(OBS_TASK_UI, [](void *opaque) {
auto &value = *static_cast<service_command_context *>(opaque);
if (obs_frontend_streaming_active()) {
value.error = "OBS is already streaming. Stop the current output before starting a private test.";
return;
}
obs_service_t *existing = obs_frontend_get_streaming_service();
if (!existing) {
value.error = "OBS does not have a streaming service to restore after the test.";
return;
}
const char *type_id = obs_service_get_id(existing);
obs_data_t *settings = obs_service_get_settings(existing);
value.snapshot_type = type_id ? type_id : "";
value.snapshot_settings = settings ? obs_data_get_json(settings) : "";
if (settings) obs_data_release(settings);
// The frontend owns this borrowed service reference.
value.ok = !value.snapshot_type.empty() && !value.snapshot_settings.empty();
if (!value.ok) value.error = "The current OBS stream service could not be snapshotted safely.";
}, &context, true);
return json{{"type", "stream_test_service_state"}, {"protocol_version", protocol_version},
{"request_id", message.value("request_id", "")}, {"state", context.ok ? "snapshotted" : "failed"},
{"error", context.error}, {"snapshot", {{"service_type", context.snapshot_type}, {"settings_json", context.snapshot_settings}}}};
} else if (type == "stream_test_begin" && message.contains("payload")) {
const auto &payload = message["payload"];
service_command_context context;
context.begin = true;
context.server = payload.value("server", "");
context.key = payload.value("key", "");
context.session_id = payload.value("session_id", "");
if ((context.server.rfind("rtmp://", 0) != 0 && context.server.rfind("rtmps://", 0) != 0) ||
context.server.size() > 1000 || context.key.empty() || context.key.size() > 1000 || context.session_id.size() > 80) {
context.error = "The private ingest destination is invalid.";
} else {
obs_queue_task(OBS_TASK_UI, apply_stream_test_service, &context, true);
}
return json{{"type", "stream_test_service_state"}, {"protocol_version", protocol_version},
{"request_id", message.value("request_id", "")}, {"state", context.ok ? "started" : "failed"},
{"session_id", context.session_id}, {"error", context.error}};
} else if (type == "stream_test_restore" && message.contains("payload")) {
const auto &payload = message["payload"];
service_command_context context;
context.restore_type = payload.value("service_type", "");
context.restore_settings = payload.value("settings_json", "");
context.session_id = payload.value("session_id", "");
if (context.restore_type.empty() || context.restore_type.size() > 200 || context.restore_settings.empty() ||
context.restore_settings.size() > max_json_bytes / 2) {
context.error = "The OBS recovery snapshot is incomplete.";
} else {
obs_queue_task(OBS_TASK_UI, apply_stream_test_service, &context, true);
}
return json{{"type", "stream_test_service_state"}, {"protocol_version", protocol_version},
{"request_id", message.value("request_id", "")}, {"state", context.ok ? "restored" : "failed"},
{"session_id", context.session_id}, {"error", context.error}};
}
return std::nullopt;
}
@ -373,7 +682,7 @@ static bool read_available_command(HANDLE pipe)
const auto response = handle_command(json::parse(body.begin(), body.end()));
if (response && !write_json(pipe, *response)) return false;
}
catch (const std::exception &error) { blog(LOG_WARNING, "[Lumi Companion] Ignored invalid IPC command: %s", error.what()); }
catch (const std::exception &error) { blog(LOG_WARNING, "[Lumi Companion] event=ipc_command_rejected Ignored invalid IPC command: %s", error.what()); }
return true;
}
@ -394,11 +703,14 @@ static void run_pipe_worker()
worker_signal.wait_for(lock, std::chrono::seconds(2));
continue;
}
blog(LOG_INFO, "[Lumi Companion] connected to the same-user Companion IPC endpoint");
bool connected = write_json(pipe, {{"type", "hello"}, {"protocol_version", protocol_version}, {"obs_version", obs_get_version_string()}, {"bridge_version", bridge_version}});
blog(LOG_INFO, "[Lumi Companion] event=ipc_connected connected to the same-user Companion IPC endpoint");
bool connected = write_json(pipe, {{"type", "hello"}, {"protocol_version", protocol_version},
{"obs_version", obs_get_version_string()}, {"bridge_version", bridge_version},
{"obs_process_id", static_cast<uint32_t>(GetCurrentProcessId())}});
source_list_dirty.store(true, std::memory_order_release);
obs_state_dirty.store(true, std::memory_order_release);
auto health_sent = std::chrono::steady_clock::now();
auto stream_metrics_sent = std::chrono::steady_clock::now();
while (connected && !stopping.load(std::memory_order_acquire)) {
connected = read_available_command(pipe);
if (connected && source_list_dirty.exchange(false, std::memory_order_acq_rel)) connected = write_json(pipe, source_list_message());
@ -415,11 +727,16 @@ static void run_pipe_worker()
{"status", "healthy"}, {"audio_frames_dropped", audio_queue.dropped()}});
health_sent = now;
}
if (connected && now - stream_metrics_sent >= std::chrono::seconds(1)) {
const auto metrics = stream_test_metrics_message();
if (!metrics.empty()) connected = write_json(pipe, metrics);
stream_metrics_sent = now;
}
std::unique_lock lock(worker_signal_mutex);
worker_signal.wait_for(lock, std::chrono::milliseconds(10));
}
CloseHandle(pipe);
blog(LOG_INFO, "[Lumi Companion] disconnected from Companion IPC; retrying safely");
blog(LOG_INFO, "[Lumi Companion] event=ipc_disconnected disconnected from Companion IPC; retrying safely");
}
}
@ -455,13 +772,13 @@ bool obs_module_load(void)
const char *version = obs_get_version_string();
const int major = version ? std::atoi(version) : 0;
if (major < 31) {
blog(LOG_ERROR, "[Lumi Companion] OBS %s is unsupported; version 31 or newer is required", version ? version : "unknown");
blog(LOG_ERROR, "[Lumi Companion] event=obs_version_unsupported OBS %s is unsupported; version 31 or newer is required", version ? version : "unknown");
return false;
}
stopping.store(false, std::memory_order_release);
obs_frontend_add_event_callback(frontend_event, nullptr);
worker_thread = std::thread(run_pipe_worker);
blog(LOG_INFO, "[Lumi Companion] bridge %s loaded", bridge_version);
blog(LOG_INFO, "[Lumi Companion] event=bridge_loaded bridge %s loaded", bridge_version);
return true;
}
@ -469,8 +786,17 @@ void obs_module_unload(void)
{
obs_frontend_remove_event_callback(frontend_event, nullptr);
detach_source();
{
std::scoped_lock lock(stream_test_mutex);
if (stream_test_output) {
if (obs_output_active(stream_test_output)) obs_output_force_stop(stream_test_output);
obs_output_release(stream_test_output);
stream_test_output = nullptr;
}
stream_test_session.clear();
}
stopping.store(true, std::memory_order_release);
worker_signal.notify_all();
if (worker_thread.joinable()) worker_thread.join();
blog(LOG_INFO, "[Lumi Companion] bridge unloaded");
blog(LOG_INFO, "[Lumi Companion] event=bridge_unloaded bridge unloaded");
}

View File

@ -0,0 +1,40 @@
namespace Lumi.Companion.Overlay;
public readonly record struct PhysicalRect(int X, int Y, int Width, int Height);
public static class AnchorLayout
{
public static void ClampPadding(OverlayContainerSettings settings)
{
var top = settings.Anchor is OverlayAnchor.TopLeft or OverlayAnchor.Top or OverlayAnchor.TopRight or OverlayAnchor.Left or OverlayAnchor.Right;
var bottom = settings.Anchor is OverlayAnchor.BottomLeft or OverlayAnchor.Bottom or OverlayAnchor.BottomRight or OverlayAnchor.Left or OverlayAnchor.Right;
var left = settings.Anchor is OverlayAnchor.TopLeft or OverlayAnchor.Left or OverlayAnchor.BottomLeft or OverlayAnchor.Top or OverlayAnchor.Bottom;
var right = settings.Anchor is OverlayAnchor.TopRight or OverlayAnchor.Right or OverlayAnchor.BottomRight or OverlayAnchor.Top or OverlayAnchor.Bottom;
if (!top) settings.PaddingTop = 0;
if (!right) settings.PaddingRight = 0;
if (!bottom) settings.PaddingBottom = 0;
if (!left) settings.PaddingLeft = 0;
}
public static PhysicalRect Calculate(OverlayContainerSettings value, int monitorWidth, int monitorHeight)
{
ClampPadding(value);
var availableWidth = Math.Max(1, monitorWidth - value.PaddingLeft - value.PaddingRight);
var availableHeight = Math.Max(1, monitorHeight - value.PaddingTop - value.PaddingBottom);
var width = Math.Clamp(value.Width, 1, availableWidth);
var height = Math.Clamp(value.Height, 1, availableHeight);
var x = value.Anchor switch
{
OverlayAnchor.TopRight or OverlayAnchor.Right or OverlayAnchor.BottomRight => monitorWidth - value.PaddingRight - width,
OverlayAnchor.Top or OverlayAnchor.Bottom => value.PaddingLeft + (availableWidth - width) / 2,
_ => value.PaddingLeft
};
var y = value.Anchor switch
{
OverlayAnchor.BottomLeft or OverlayAnchor.Bottom or OverlayAnchor.BottomRight => monitorHeight - value.PaddingBottom - height,
OverlayAnchor.Left or OverlayAnchor.Right => value.PaddingTop + (availableHeight - height) / 2,
_ => value.PaddingTop
};
return new(x, y, width, height);
}
}

View File

@ -0,0 +1,105 @@
using System.Runtime.InteropServices;
namespace Lumi.Companion.Overlay;
public sealed class GlobalHotkeyManager : IDisposable
{
private const uint WmHotkey = 0x0312;
private const uint WmQuit = 0x0012;
private readonly Dictionary<int, Action> _actions = [];
private readonly ManualResetEventSlim _ready = new();
private readonly Thread _thread;
private IntPtr _window;
private uint _threadId;
private int _nextId = 0x4C00;
public GlobalHotkeyManager()
{
_thread = new Thread(MessageLoop) { IsBackground = true, Name = "Lumi Overlay hotkeys" };
_thread.Start();
_ready.Wait(TimeSpan.FromSeconds(3));
}
public bool Register(string chord, Action action)
{
if (!TryParse(chord, out var modifiers, out var key) || _window == IntPtr.Zero) return false;
var id = Interlocked.Increment(ref _nextId);
if (!RegisterHotKey(_window, id, modifiers | 0x4000, key)) return false;
lock (_actions) _actions[id] = action;
return true;
}
public void Clear()
{
lock (_actions)
{
foreach (var id in _actions.Keys) UnregisterHotKey(_window, id);
_actions.Clear();
}
}
private void MessageLoop()
{
_threadId = GetCurrentThreadId();
_window = CreateWindowEx(0, "STATIC", "LumiOverlayHotkeys", 0, 0, 0, 0, 0, new IntPtr(-3), IntPtr.Zero, IntPtr.Zero, IntPtr.Zero);
_ready.Set();
while (GetMessage(out var message, IntPtr.Zero, 0, 0) > 0)
{
if (message.message == WmHotkey)
{
Action? action;
lock (_actions) _actions.TryGetValue(message.wParam.ToInt32(), out action);
try { action?.Invoke(); } catch { }
}
TranslateMessage(ref message);
DispatchMessage(ref message);
}
}
internal static bool TryParse(string chord, out uint modifiers, out uint key)
{
modifiers = 0;
key = 0;
var parts = (chord ?? string.Empty).Split('+', StringSplitOptions.TrimEntries | StringSplitOptions.RemoveEmptyEntries);
foreach (var part in parts)
{
if (part.Equals("Ctrl", StringComparison.OrdinalIgnoreCase)) modifiers |= 0x0002;
else if (part.Equals("Alt", StringComparison.OrdinalIgnoreCase)) modifiers |= 0x0001;
else if (part.Equals("Shift", StringComparison.OrdinalIgnoreCase)) modifiers |= 0x0004;
else if (part.Equals("Win", StringComparison.OrdinalIgnoreCase)) modifiers |= 0x0008;
else if (part.Length == 1 && char.IsAsciiLetterOrDigit(part[0])) key = char.ToUpperInvariant(part[0]);
else if (part.StartsWith('F') && int.TryParse(part[1..], out var function) && function is >= 1 and <= 24) key = (uint)(0x70 + function - 1);
else if (VirtualKeys.TryGetValue(part, out var virtualKey)) key = virtualKey;
else return false;
}
return key != 0;
}
private static readonly Dictionary<string, uint> VirtualKeys = new(StringComparer.OrdinalIgnoreCase)
{
["Backspace"] = 0x08, ["Tab"] = 0x09, ["Enter"] = 0x0D, ["Pause"] = 0x13,
["CapsLock"] = 0x14, ["Escape"] = 0x1B, ["Space"] = 0x20, ["PageUp"] = 0x21,
["PageDown"] = 0x22, ["End"] = 0x23, ["Home"] = 0x24, ["Left"] = 0x25,
["Up"] = 0x26, ["Right"] = 0x27, ["Down"] = 0x28, ["Insert"] = 0x2D,
["Delete"] = 0x2E, ["NumLock"] = 0x90, ["ScrollLock"] = 0x91
};
public void Dispose()
{
Clear();
if (_threadId != 0) PostThreadMessage(_threadId, WmQuit, IntPtr.Zero, IntPtr.Zero);
_thread.Join(TimeSpan.FromSeconds(1));
_ready.Dispose();
}
[StructLayout(LayoutKind.Sequential)]
private struct Msg { public IntPtr hwnd; public uint message; public IntPtr wParam; public IntPtr lParam; public uint time; public int x; public int y; }
[DllImport("user32.dll", SetLastError = true)] private static extern bool RegisterHotKey(IntPtr hwnd, int id, uint modifiers, uint key);
[DllImport("user32.dll", SetLastError = true)] private static extern bool UnregisterHotKey(IntPtr hwnd, int id);
[DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Unicode)] private static extern IntPtr CreateWindowEx(uint exStyle, string className, string windowName, uint style, int x, int y, int width, int height, IntPtr parent, IntPtr menu, IntPtr instance, IntPtr parameter);
[DllImport("user32.dll")] private static extern int GetMessage(out Msg message, IntPtr hwnd, uint min, uint max);
[DllImport("user32.dll")] private static extern bool TranslateMessage(ref Msg message);
[DllImport("user32.dll")] private static extern IntPtr DispatchMessage(ref Msg message);
[DllImport("kernel32.dll")] private static extern uint GetCurrentThreadId();
[DllImport("user32.dll")] private static extern bool PostThreadMessage(uint threadId, uint message, IntPtr wParam, IntPtr lParam);
}

View File

@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net8.0-windows10.0.19041.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AssemblyName>Lumi.Companion.Overlay</AssemblyName>
<RootNamespace>Lumi.Companion.Overlay</RootNamespace>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="../../src/Lumi.Companion.Abstractions/Lumi.Companion.Abstractions.csproj" />
<PackageReference Include="Avalonia" Version="12.1.0" />
<PackageReference Include="Avalonia.Desktop" Version="12.1.0" />
<PackageReference Include="Microsoft.Win32.SystemEvents" Version="8.0.0" />
<PackageReference Include="SkiaSharp" Version="3.119.4" />
</ItemGroup>
</Project>

View File

@ -0,0 +1,292 @@
using System.Text.Json;
using Avalonia.Threading;
using Lumi.Companion.Abstractions;
namespace Lumi.Companion.Overlay;
public sealed class LumiOverlayRuntime : ICompanionPluginContribution, IAsyncDisposable
{
private const string PluginId = "lumi_overlay";
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
private readonly OverlaySettingsStore _store;
private readonly ICompanionPluginTransport _transport;
private readonly NativeOverlayWindow _window;
private readonly GlobalHotkeyManager _hotkeys;
private readonly CancellationTokenSource _lifetime = new();
private readonly List<OverlaySource> _catalog = [];
private readonly List<OverlayEventType> _eventCatalog = [];
private readonly List<OverlayFeedStatus> _feedStatuses = [];
private ICompanionPluginLiveChannel? _channel;
private Task? _feedTask;
private bool? _visibilityOverride;
private bool _preview;
private bool _initialized;
private string _hotkeyStatus = "Hotkeys are not registered yet.";
public LumiOverlayRuntime(string dataDirectory, ICompanionPluginTransport transport)
{
Directory.CreateDirectory(dataDirectory);
_store = new OverlaySettingsStore(Path.Combine(dataDirectory, "settings.json"));
_transport = transport;
_window = new NativeOverlayWindow();
_window.NativeStatusChanged += () => RaiseChanged();
_window.EditSettingsChanged += () => RaiseChanged();
_hotkeys = new GlobalHotkeyManager();
Actions =
[
new("visibility", () => VisibilityPresentation.Action, _ => { ToggleVisibilityOverride(); return Task.CompletedTask; }, () => _initialized && VisibilityPresentation.CanToggle, 10),
new("preview", () => _preview ? "End configuration preview" : "Start configuration preview", _ => { TogglePreview(); return Task.CompletedTask; }, () => _initialized, 20)
];
}
public CompanionPluginDescriptor Descriptor { get; } = new(PluginId, "Lumi Overlay", new Version(0, 1, 2),
"Native, capture-excluded monitor overlay for Lumi chat and stream events.", 150);
public IReadOnlyList<CompanionPluginPage> Pages { get; } = [new("LumiOverlay", "Overview & settings", 10)];
public IReadOnlyList<CompanionPluginAction> Actions { get; }
public CompanionPluginStatus Status { get; private set; } = new(CompanionPluginHealth.Ready, "Starting", "Loading local overlay settings.");
public LumiOverlaySettings Settings => _store.Current;
public IReadOnlyList<OverlaySource> Sources { get { lock (_catalog) return _catalog.ToList(); } }
public IReadOnlyList<OverlayEventType> AvailableEventTypes { get { lock (_catalog) return _eventCatalog.ToList(); } }
public IReadOnlyList<OverlayFeedStatus> FeedStatuses { get { lock (_catalog) return _feedStatuses.ToList(); } }
public CaptureExclusionStatus CaptureExclusion => _window.CaptureExclusion;
public string? MonitorWarning => _window.MonitorWarning;
public IReadOnlyList<(string Id, string Label)> Monitors => _window.Monitors();
public bool PreviewActive => _preview;
public bool EditModeActive { get; private set; }
public bool VisibilityOverride => _visibilityOverride.HasValue;
public bool IsInitialized => _initialized;
public string HotkeyStatus => _hotkeyStatus;
public OverlayVisibilityPresentation VisibilityPresentation =>
OverlayVisibilityPresenter.Describe(Settings, AutomaticVisibility(), _preview, _visibilityOverride);
public event Action? Changed;
public async Task InitializeAsync(CancellationToken cancellationToken = default)
{
await _store.LoadAsync(cancellationToken).ConfigureAwait(false);
_initialized = true;
await Dispatcher.UIThread.InvokeAsync(() =>
{
_window.ApplySettings(Settings);
ConfigureHotkeys();
ApplyVisibility();
});
_feedTask = _transport.RunLiveJsonAsync("/plugins/lumi_overlay/live", RunConnectedAsync, OnLiveStatus, _lifetime.Token);
RaiseChanged();
}
public Task SaveSettingsAsync(CancellationToken cancellationToken = default) =>
SaveSettingsAsync(Settings, cancellationToken);
public async Task SaveSettingsAsync(LumiOverlaySettings settings, CancellationToken cancellationToken = default)
{
await _store.SaveAsync(settings, cancellationToken).ConfigureAwait(false);
await Dispatcher.UIThread.InvokeAsync(() =>
{
_window.ApplySettings(Settings);
ConfigureHotkeys();
ApplyVisibility();
});
await SendSubscriptionAsync(cancellationToken).ConfigureAwait(false);
RaiseChanged();
}
public void ToggleVisibilityOverride()
{
if (!VisibilityPresentation.CanToggle) return;
_visibilityOverride = _visibilityOverride.HasValue ? null : !AutomaticVisibility();
Dispatcher.UIThread.Post(ApplyVisibility);
RaiseChanged();
}
public void TogglePreview()
{
_preview = !_preview;
if (_preview)
{
_window.AddChat(new OverlayFeedMessage
{
Id = $"sample-chat-{Guid.NewGuid():N}", Platform = "twitch", Text = "This is a native Lumi Overlay preview ✨",
Author = new() { Name = "Lumi viewer", Badges = [new() { Label = "SUB" }] }
});
_window.AddEvent(new($"sample-event-{Guid.NewGuid():N}", "twitch.follow", "twitch", "A new viewer followed the channel", default));
}
else _window.EndPreview();
Dispatcher.UIThread.Post(ApplyVisibility);
RaiseChanged();
}
public void BeginEdit(LumiOverlaySettings draft)
{
EditModeActive = true;
_window.BeginEdit(draft);
RaiseChanged();
}
public void EndEdit()
{
if (!EditModeActive) return;
EditModeActive = false;
_window.EndEdit();
_window.ApplySettings(Settings);
ApplyVisibility();
RaiseChanged();
}
private async Task RunConnectedAsync(ICompanionPluginLiveChannel channel, CancellationToken cancellationToken)
{
_channel = channel;
await SendSubscriptionAsync(cancellationToken).ConfigureAwait(false);
try
{
await foreach (var json in channel.ReadAllAsync(cancellationToken).ConfigureAwait(false))
{
using var document = JsonDocument.Parse(json);
var root = document.RootElement;
var type = root.TryGetProperty("type", out var typeValue) ? typeValue.GetString() : null;
if (type is "hello" or "catalog") UpdateCatalog(root);
else if (type == "chat") HandleChat(root);
else if (type == "event") HandleEvent(root);
else if (type == "ping") await channel.SendJsonAsync(new { type = "pong" }, cancellationToken).ConfigureAwait(false);
}
}
finally
{
if (ReferenceEquals(_channel, channel)) _channel = null;
}
}
private void UpdateCatalog(JsonElement root)
{
var sources = new List<OverlaySource>();
if (root.TryGetProperty("sources", out var sourceValues))
foreach (var value in sourceValues.EnumerateArray())
sources.Add(new(
Read(value, "id"), Read(value, "platform"), Read(value, "label"),
value.TryGetProperty("live", out var live) && live.ValueKind is JsonValueKind.True or JsonValueKind.False ? live.GetBoolean() : null,
!value.TryGetProperty("available", out var available) || available.GetBoolean(), Read(value, "status")));
var eventTypes = new List<OverlayEventType>();
if (root.TryGetProperty("event_types", out var eventValues))
foreach (var value in eventValues.EnumerateArray())
eventTypes.Add(new(Read(value, "id"), Read(value, "label"), Read(value, "platform")));
var statuses = new List<OverlayFeedStatus>();
if (root.TryGetProperty("statuses", out var statusValues))
foreach (var value in statusValues.EnumerateArray())
statuses.Add(new(Read(value, "platform"), Read(value, "state"), Read(value, "detail")));
lock (_catalog)
{
if (sources.Count > 0) { _catalog.Clear(); _catalog.AddRange(sources); }
if (eventTypes.Count > 0) { _eventCatalog.Clear(); _eventCatalog.AddRange(eventTypes); }
if (statuses.Count > 0) { _feedStatuses.Clear(); _feedStatuses.AddRange(statuses); }
}
Dispatcher.UIThread.Post(ApplyVisibility);
RaiseChanged();
}
private void HandleChat(JsonElement root)
{
if (!root.TryGetProperty("message", out var value)) return;
var message = value.Deserialize<OverlayFeedMessage>(JsonOptions);
if (message is null || string.IsNullOrWhiteSpace(message.Id)) return;
Dispatcher.UIThread.Post(() => _window.AddChat(message));
}
private void HandleEvent(JsonElement root)
{
if (!root.TryGetProperty("event", out var value)) return;
var id = Read(value, "id");
var type = Read(value, "type");
if (string.IsNullOrWhiteSpace(id) || string.IsNullOrWhiteSpace(type)) return;
var platform = type.Split('.')[0];
var payload = value.TryGetProperty("payload", out var payloadValue) ? payloadValue.Clone() : default;
var summary = EventSummary(type, payload);
Dispatcher.UIThread.Post(() => _window.AddEvent(new(id, type, platform, summary, payload)));
}
private async Task SendSubscriptionAsync(CancellationToken cancellationToken = default)
{
var channel = _channel;
if (channel is null) return;
await channel.SendJsonAsync(new
{
type = "subscribe",
sources = Settings.SelectedSourceIds.ToArray(),
events = Settings.EventTypes.ToArray()
}, cancellationToken).ConfigureAwait(false);
}
private void ApplyVisibility()
{
_window.SetRequestedVisible(EditModeActive || OverlayVisibilityPolicy.Effective(Settings, Sources, _preview, _visibilityOverride));
}
private bool AutomaticVisibility() => OverlayVisibilityPolicy.Automatic(Settings, Sources);
private void ConfigureHotkeys()
{
_hotkeys.Clear();
var visibility = _hotkeys.Register(Settings.ToggleVisibilityHotkey, () => Dispatcher.UIThread.Post(ToggleVisibilityOverride));
var preview = _hotkeys.Register(Settings.TogglePreviewHotkey, () => Dispatcher.UIThread.Post(TogglePreview));
_hotkeyStatus = visibility && preview
? "Both global hotkeys are active."
: !visibility && !preview
? "Neither hotkey could be registered. Check the format or choose shortcuts not reserved by another app."
: visibility
? "Visibility hotkey is active; the preview shortcut is invalid or reserved by another app."
: "Preview hotkey is active; the visibility shortcut is invalid or reserved by another app.";
}
private void OnLiveStatus(CompanionPluginLiveStatus value)
{
Status = value.State switch
{
CompanionPluginLiveState.Connected => new(CompanionPluginHealth.Healthy, "Feed connected", value.Detail),
CompanionPluginLiveState.WaitingForPairing => new(CompanionPluginHealth.Warning, "Pairing required", value.Detail),
CompanionPluginLiveState.Reconnecting => new(CompanionPluginHealth.Warning, "Reconnecting", value.Detail),
CompanionPluginLiveState.Stopped when _lifetime.IsCancellationRequested => new(CompanionPluginHealth.Ready, "Stopped", value.Detail),
_ => new(CompanionPluginHealth.Ready, "Connecting", value.Detail)
};
RaiseChanged();
}
private static string EventSummary(string type, JsonElement payload)
{
string Field(params string[] names)
{
foreach (var name in names)
if (payload.ValueKind == JsonValueKind.Object && payload.TryGetProperty(name, out var value))
return value.ValueKind == JsonValueKind.String ? value.GetString() ?? "" : value.ToString();
return "";
}
var user = Field("user_name", "member_name", "author_name");
return type switch
{
"twitch.follow" => $"{user.Or("A viewer")} followed",
"twitch.raid" => $"{user.Or("A channel")} raided with {Field("viewers").Or("viewers")}",
"twitch.subscribe" => $"{user.Or("A viewer")} subscribed",
"twitch.subscription_gift" => $"{user.Or("A viewer")} gifted {Field("total").Or("subscriptions")}",
"twitch.cheer" => $"{user.Or("A viewer")} cheered {Field("bits").Or("bits")}",
"twitch.channel_points" => $"{user.Or("A viewer")} redeemed {Field("reward_title").Or("a reward")}",
"discord.member_join" => $"{user.Or("A member")} joined Discord",
_ => $"{type.Replace('.', ' ')} · {user}".TrimEnd(' ', '·')
};
}
private static string Read(JsonElement value, string property) =>
value.TryGetProperty(property, out var found) && found.ValueKind == JsonValueKind.String ? found.GetString() ?? "" : "";
private void RaiseChanged() => Changed?.Invoke();
public async ValueTask DisposeAsync()
{
_lifetime.Cancel();
if (_feedTask is not null) try { await _feedTask.ConfigureAwait(false); } catch (OperationCanceledException) { }
await Dispatcher.UIThread.InvokeAsync(async () => await _window.DisposeAsync());
_hotkeys.Dispose();
_lifetime.Dispose();
}
}
internal static class OverlayStringExtensions
{
public static string Or(this string value, string fallback) => string.IsNullOrWhiteSpace(value) ? fallback : value;
}

View File

@ -0,0 +1,837 @@
using System.Collections.Concurrent;
using System.Runtime.InteropServices;
using Avalonia;
using Avalonia.Animation;
using Avalonia.Animation.Easings;
using Avalonia.Controls;
using Avalonia.Controls.ApplicationLifetimes;
using Avalonia.Input;
using Avalonia.Media;
using Avalonia.Media.Imaging;
using Avalonia.Platform;
using Avalonia.Threading;
using Microsoft.Win32;
using SkiaSharp;
namespace Lumi.Companion.Overlay;
public enum CaptureExclusionState { Pending, Active, Unsupported, Failed }
public sealed record CaptureExclusionStatus(CaptureExclusionState State, string Detail);
public sealed class NativeOverlayWindow : Window, IAsyncDisposable
{
private const int GwlExStyle = -20;
private const long WsExTransparent = 0x20;
private const long WsExToolWindow = 0x80;
private const long WsExLayered = 0x80000;
private const long WsExNoActivate = 0x08000000;
private const uint SwpNoActivate = 0x0010;
private const uint SwpFrameChanged = 0x0020;
private const uint WdaExcludeFromCapture = 0x11;
private static readonly IntPtr HwndTopmost = new(-1);
private readonly Grid _root = new();
private readonly Canvas _surface = new();
private readonly Grid _unifiedHolder = new();
private readonly Grid _chatHolder = new();
private readonly Grid _eventHolder = new();
private readonly StackPanel _unifiedPanel = new() { Spacing = 8 };
private readonly StackPanel _chatPanel = new() { Spacing = 8 };
private readonly StackPanel _eventPanel = new() { Spacing = 8 };
private readonly Border _unifiedEditor;
private readonly Border _chatEditor;
private readonly Border _eventEditor;
private readonly Dictionary<string, CardVisual> _cards = [];
private readonly DispatcherTimer _timer;
private readonly HttpsImageCache _images = new();
private LumiOverlaySettings _settings = new();
private OverlayLifecycleQueue _chat;
private OverlayLifecycleQueue _events;
private bool _disposed;
private bool _visibleRequested;
private string? _temporaryMonitorWarning;
private bool _editing;
private Border? _dragEditor;
private OverlayContainerSettings? _dragSettings;
private Point _dragOrigin;
private PhysicalRect _dragRect;
private bool _resizing;
public NativeOverlayWindow()
{
WindowDecorations = Avalonia.Controls.WindowDecorations.None;
TransparencyLevelHint = [WindowTransparencyLevel.Transparent];
Background = Brushes.Transparent;
Topmost = true;
ShowInTaskbar = false;
ShowActivated = false;
CanResize = false;
Focusable = false;
Content = _root;
_root.Children.Add(_surface);
_unifiedHolder.Children.Add(_unifiedPanel);
_chatHolder.Children.Add(_chatPanel);
_eventHolder.Children.Add(_eventPanel);
_surface.Children.Add(_unifiedHolder);
_surface.Children.Add(_chatHolder);
_surface.Children.Add(_eventHolder);
_unifiedEditor = CreateEditor("Overlay container", "unified");
_chatEditor = CreateEditor("Chat", "chat");
_eventEditor = CreateEditor("Events", "events");
_surface.Children.Add(_unifiedEditor);
_surface.Children.Add(_chatEditor);
_surface.Children.Add(_eventEditor);
_chat = new(() => ContainerFor("chat"));
_events = new(() => ContainerFor("event"));
_chat.Changed += Render;
_events.Changed += Render;
Opened += (_, _) => { ApplyMonitorAndStyles(); Render(); };
Screens.Changed += OnScreensChanged;
SystemEvents.DisplaySettingsChanged += OnSystemDisplayChanged;
SystemEvents.PowerModeChanged += OnPowerChanged;
SystemEvents.SessionSwitch += OnSessionSwitch;
_timer = new DispatcherTimer(TimeSpan.FromMilliseconds(33), DispatcherPriority.Render, (_, _) =>
{
_chat.Advance();
_events.Advance();
});
_timer.Start();
}
public CaptureExclusionStatus CaptureExclusion { get; private set; } = new(CaptureExclusionState.Pending, "Waiting for the native overlay window.");
public string? MonitorWarning => _temporaryMonitorWarning;
public event Action? NativeStatusChanged;
public event Action? EditSettingsChanged;
public void ApplySettings(LumiOverlaySettings settings)
{
_settings = settings;
_unifiedPanel.Children.Clear();
_chatPanel.Children.Clear();
_eventPanel.Children.Clear();
_cards.Clear();
ApplyMonitorAndStyles();
Render();
}
public void SetRequestedVisible(bool visible)
{
_visibleRequested = visible;
if (visible)
{
if (!IsVisible) Show();
ApplyMonitorAndStyles();
}
else if (IsVisible) Hide();
}
public void BeginEdit(LumiOverlaySettings settings)
{
_editing = true;
ApplySettings(settings);
if (!_chat.Items.Any(item => item.Id.StartsWith("edit-sample-", StringComparison.Ordinal)))
{
AddChat(new OverlayFeedMessage
{
Id = $"edit-sample-chat-{Guid.NewGuid():N}",
Platform = "twitch",
Text = "Drag this container to move it · drag the lower-right corner to resize",
Author = new() { Name = "Lumi preview", Badges = [new() { Label = "LIVE" }] }
});
AddEvent(new($"edit-sample-event-{Guid.NewGuid():N}", "twitch.follow", "twitch", "New follower preview", default));
}
SetRequestedVisible(true);
ApplyMonitorAndStyles();
Activate();
}
public void EndEdit()
{
if (!_editing) return;
_editing = false;
Clear();
ApplyMonitorAndStyles();
Render();
}
public void AddChat(OverlayFeedMessage message) => _chat.Add(message.Id, "chat", message);
public void AddEvent(OverlayRenderedEvent value) => _events.Add(value.Id, "event", value);
public void Clear() { _chat.Clear(); _events.Clear(); }
public void EndPreview()
{
_chat.ExitWhere(item => item.Id.StartsWith("sample-", StringComparison.Ordinal));
_events.ExitWhere(item => item.Id.StartsWith("sample-", StringComparison.Ordinal));
}
private OverlayContainerSettings ContainerFor(string kind) =>
_settings.Layout == OverlayLayoutMode.Unified ? _settings.Unified : kind == "chat" ? _settings.Chat : _settings.Events;
private void Render()
{
if (!Dispatcher.UIThread.CheckAccess()) { Dispatcher.UIThread.Post(Render); return; }
var screen = ResolveScreen();
if (screen is null) return;
var width = screen.Bounds.Width;
var height = screen.Bounds.Height;
var scaling = screen.Scaling;
if (_settings.Layout == OverlayLayoutMode.Unified)
{
var combined = _chat.Items.Concat(_events.Items).OrderBy(item => item.QueuedAt).ToList();
ConfigureContainer(_unifiedHolder, _unifiedPanel, combined, _settings.Unified,
Logical(AnchorLayout.Calculate(_settings.Unified, width, height), scaling), true);
ConfigureContainer(_chatHolder, _chatPanel, [], _settings.Chat, default, false);
ConfigureContainer(_eventHolder, _eventPanel, [], _settings.Events, default, false);
}
else if (_settings.Layout is OverlayLayoutMode.DividedHorizontal or OverlayLayoutMode.DividedVertical)
{
var rect = AnchorLayout.Calculate(_settings.Unified, width, height);
if (_settings.Layout == OverlayLayoutMode.DividedHorizontal)
{
var chatWidth = rect.Width * _settings.ChatPercentage / 100;
ConfigureContainer(_chatHolder, _chatPanel, _chat.Items, _settings.Chat, Logical(new(rect.X, rect.Y, chatWidth, rect.Height), scaling), true);
ConfigureContainer(_eventHolder, _eventPanel, _events.Items, _settings.Events, Logical(new(rect.X + chatWidth, rect.Y, rect.Width - chatWidth, rect.Height), scaling), true);
}
else
{
var chatHeight = rect.Height * _settings.ChatPercentage / 100;
ConfigureContainer(_chatHolder, _chatPanel, _chat.Items, _settings.Chat, Logical(new(rect.X, rect.Y, rect.Width, chatHeight), scaling), true);
ConfigureContainer(_eventHolder, _eventPanel, _events.Items, _settings.Events, Logical(new(rect.X, rect.Y + chatHeight, rect.Width, rect.Height - chatHeight), scaling), true);
}
ConfigureContainer(_unifiedHolder, _unifiedPanel, [], _settings.Unified, default, false);
}
else
{
ConfigureContainer(_unifiedHolder, _unifiedPanel, [], _settings.Unified, default, false);
ConfigureContainer(_chatHolder, _chatPanel, _chat.Items, _settings.Chat, Logical(AnchorLayout.Calculate(_settings.Chat, width, height), scaling), true);
ConfigureContainer(_eventHolder, _eventPanel, _events.Items, _settings.Events, Logical(AnchorLayout.Calculate(_settings.Events, width, height), scaling), true);
}
var activeIds = _chat.Items.Concat(_events.Items).Select(item => item.Id).ToHashSet();
foreach (var stale in _cards.Keys.Where(id => !activeIds.Contains(id)).ToList()) _cards.Remove(stale);
RenderEditors(width, height, scaling);
}
private Border CreateEditor(string label, string tag)
{
var editor = new Border
{
Tag = tag,
BorderBrush = Brush.Parse("#28B7C8"),
BorderThickness = new Thickness(3),
Background = Brush.Parse("#082C3333"),
CornerRadius = new CornerRadius(8),
Child = new Border
{
Background = Brush.Parse("#D9121212"),
CornerRadius = new CornerRadius(6),
Padding = new Thickness(9, 5),
HorizontalAlignment = Avalonia.Layout.HorizontalAlignment.Left,
VerticalAlignment = Avalonia.Layout.VerticalAlignment.Top,
Child = new TextBlock { Text = label, Foreground = Brushes.White, FontWeight = FontWeight.SemiBold, FontSize = 12 }
},
Cursor = new Cursor(StandardCursorType.SizeAll),
IsVisible = false
};
editor.PointerPressed += OnEditorPointerPressed;
editor.PointerMoved += OnEditorPointerMoved;
editor.PointerReleased += OnEditorPointerReleased;
return editor;
}
private void RenderEditors(int width, int height, double scaling)
{
if (!_editing)
{
_unifiedEditor.IsVisible = _chatEditor.IsVisible = _eventEditor.IsVisible = false;
return;
}
if (_settings.Layout is OverlayLayoutMode.Unified or OverlayLayoutMode.DividedHorizontal or OverlayLayoutMode.DividedVertical)
{
ConfigureEditor(_unifiedEditor, Logical(AnchorLayout.Calculate(_settings.Unified, width, height), scaling), true);
ConfigureEditor(_chatEditor, default, false);
ConfigureEditor(_eventEditor, default, false);
}
else
{
ConfigureEditor(_unifiedEditor, default, false);
ConfigureEditor(_chatEditor, Logical(AnchorLayout.Calculate(_settings.Chat, width, height), scaling), true);
ConfigureEditor(_eventEditor, Logical(AnchorLayout.Calculate(_settings.Events, width, height), scaling), true);
}
}
private static void ConfigureEditor(Border editor, PhysicalRect rect, bool visible)
{
editor.IsVisible = visible;
if (!visible) return;
editor.Width = rect.Width;
editor.Height = rect.Height;
Canvas.SetLeft(editor, rect.X);
Canvas.SetTop(editor, rect.Y);
}
private void OnEditorPointerPressed(object? sender, PointerPressedEventArgs args)
{
if (!_editing || sender is not Border editor) return;
var screen = ResolveScreen();
if (screen is null) return;
var tag = editor.Tag?.ToString();
_dragSettings = tag == "chat" ? _settings.Chat : tag == "events" ? _settings.Events : _settings.Unified;
_dragRect = AnchorLayout.Calculate(_dragSettings, screen.Bounds.Width, screen.Bounds.Height);
_dragOrigin = args.GetPosition(_surface);
var position = args.GetPosition(editor);
_resizing = position.X >= editor.Bounds.Width - 24 && position.Y >= editor.Bounds.Height - 24;
_dragEditor = editor;
editor.Cursor = new Cursor(_resizing ? StandardCursorType.BottomRightCorner : StandardCursorType.SizeAll);
args.Pointer.Capture(editor);
args.Handled = true;
}
private void OnEditorPointerMoved(object? sender, PointerEventArgs args)
{
if (_dragEditor is null || _dragSettings is null) return;
var screen = ResolveScreen();
if (screen is null) return;
var current = args.GetPosition(_surface);
var dx = (int)Math.Round((current.X - _dragOrigin.X) * screen.Scaling);
var dy = (int)Math.Round((current.Y - _dragOrigin.Y) * screen.Scaling);
if (_resizing)
{
_dragSettings.Width = Math.Clamp(_dragRect.Width + dx, 160, screen.Bounds.Width);
_dragSettings.Height = Math.Clamp(_dragRect.Height + dy, 100, screen.Bounds.Height);
}
else
{
_dragSettings.Anchor = OverlayAnchor.TopLeft;
_dragSettings.PaddingLeft = Math.Clamp(_dragRect.X + dx, 0, Math.Max(0, screen.Bounds.Width - _dragSettings.Width));
_dragSettings.PaddingTop = Math.Clamp(_dragRect.Y + dy, 0, Math.Max(0, screen.Bounds.Height - _dragSettings.Height));
_dragSettings.PaddingRight = 0;
_dragSettings.PaddingBottom = 0;
}
Render();
args.Handled = true;
}
private void OnEditorPointerReleased(object? sender, PointerReleasedEventArgs args)
{
if (_dragEditor is null) return;
args.Pointer.Capture(null);
_dragEditor.Cursor = new Cursor(StandardCursorType.SizeAll);
_dragEditor = null;
_dragSettings = null;
EditSettingsChanged?.Invoke();
args.Handled = true;
}
private void ConfigureContainer(Grid holder, StackPanel panel, IEnumerable<OverlayQueueItem> values,
OverlayContainerSettings settings, PhysicalRect rect, bool visible)
{
holder.IsVisible = visible;
if (!visible) { panel.Children.Clear(); return; }
holder.Width = rect.Width;
holder.Height = rect.Height;
panel.Width = rect.Width;
panel.MaxHeight = rect.Height;
panel.HorizontalAlignment = settings.HorizontalAlignment switch
{
OverlayHorizontalAlignment.Left => Avalonia.Layout.HorizontalAlignment.Left,
OverlayHorizontalAlignment.Middle => Avalonia.Layout.HorizontalAlignment.Center,
_ => Avalonia.Layout.HorizontalAlignment.Right
};
panel.VerticalAlignment = settings.VerticalAlignment switch
{
OverlayVerticalAlignment.Top => Avalonia.Layout.VerticalAlignment.Top,
OverlayVerticalAlignment.Middle => Avalonia.Layout.VerticalAlignment.Center,
_ => Avalonia.Layout.VerticalAlignment.Bottom
};
Canvas.SetLeft(holder, rect.X);
Canvas.SetTop(holder, rect.Y);
var items = (settings.NewestDirection == OverlayNewestDirection.End ? values : values.Reverse()).ToList();
var oldPositions = panel.Children.OfType<Border>().Where(card => card.Tag is CardVisual)
.ToDictionary(card => ((CardVisual)card.Tag!).Id, card => rect.Y + card.Bounds.Y);
foreach (var other in new[] { _unifiedPanel, _chatPanel, _eventPanel })
if (!ReferenceEquals(other, panel))
foreach (var card in other.Children.OfType<Border>().Where(card => card.Tag is CardVisual visual && items.Any(item => item.Id == visual.Id)).ToList())
other.Children.Remove(card);
panel.Children.Clear();
foreach (var item in items)
{
var visual = GetOrCreateCard(item, settings, out var created);
visual.Card.MaxWidth = rect.Width;
panel.Children.Add(visual.Card);
if (!created) ApplyCardState(visual, item, settings);
}
Dispatcher.UIThread.Post(() =>
{
foreach (var visual in panel.Children.OfType<Border>().Select(card => card.Tag as CardVisual).Where(value => value is not null))
{
if (!oldPositions.TryGetValue(visual!.Id, out var old)) continue;
var target = visual.Translate.Y;
var transitions = visual.Translate.Transitions;
visual.Translate.Transitions = null;
visual.Translate.Y += old - (rect.Y + visual.Card.Bounds.Y);
Dispatcher.UIThread.Post(() =>
{
visual.Translate.Transitions = transitions;
visual.Translate.Y = target;
}, DispatcherPriority.Render);
}
}, DispatcherPriority.Render);
}
private CardVisual GetOrCreateCard(OverlayQueueItem item, OverlayContainerSettings settings, out bool created)
{
if (_cards.TryGetValue(item.Id, out var existing)) { created = false; return existing; }
var content = item.Payload switch
{
OverlayFeedMessage message => CreateChatContent(message, settings),
OverlayRenderedEvent value => CreateEventContent(value, settings),
_ => new TextBlock { Text = item.Payload.ToString() ?? "" }
};
var translate = new TranslateTransform();
var scale = new ScaleTransform();
var transforms = new TransformGroup();
transforms.Children.Add(scale);
transforms.Children.Add(translate);
var border = new Border
{
Background = OverlayColor.BrushFromRgba(settings.CardBackground),
CornerRadius = new CornerRadius(9),
Padding = new Thickness(12, 9),
Child = content,
HorizontalAlignment = settings.HorizontalAlignment switch
{
OverlayHorizontalAlignment.Left => Avalonia.Layout.HorizontalAlignment.Left,
OverlayHorizontalAlignment.Middle => Avalonia.Layout.HorizontalAlignment.Center,
_ => Avalonia.Layout.HorizontalAlignment.Right
},
RenderTransform = transforms,
RenderTransformOrigin = RelativePoint.Center
};
var visual = new CardVisual(item.Id, border, translate, scale);
border.Tag = visual;
_cards[item.Id] = visual;
created = true;
SetInitialState(visual, settings.EntryAnimation, settings);
Dispatcher.UIThread.Post(() => ApplyCardState(visual, item, settings), DispatcherPriority.Render);
return visual;
}
private static void SetInitialState(CardVisual visual, OverlayAnimation animation, OverlayContainerSettings settings)
{
visual.Card.Opacity = animation is OverlayAnimation.Fade or OverlayAnimation.FadeSlide or OverlayAnimation.ScaleFade ? 0 : 1;
visual.Translate.X = animation is OverlayAnimation.Slide or OverlayAnimation.FadeSlide ? settings.SlideDistance : 0;
visual.Scale.ScaleX = visual.Scale.ScaleY = animation == OverlayAnimation.ScaleFade ? settings.ScaleAmount : 1;
}
private static void ApplyCardState(CardVisual visual, OverlayQueueItem item, OverlayContainerSettings settings)
{
var exiting = item.State == OverlayItemState.Exiting;
var animation = exiting ? settings.ExitAnimation : settings.EntryAnimation;
var duration = TimeSpan.FromMilliseconds(exiting ? settings.ExitDurationMs : settings.EntryDurationMs);
var easing = EasingFor(settings.Easing);
visual.Card.Transitions = [new DoubleTransition { Property = OpacityProperty, Duration = duration, Easing = easing }];
visual.Translate.Transitions =
[
new DoubleTransition { Property = TranslateTransform.XProperty, Duration = duration, Easing = easing },
new DoubleTransition { Property = TranslateTransform.YProperty, Duration = duration, Easing = easing }
];
visual.Scale.Transitions =
[
new DoubleTransition { Property = ScaleTransform.ScaleXProperty, Duration = duration, Easing = easing },
new DoubleTransition { Property = ScaleTransform.ScaleYProperty, Duration = duration, Easing = easing }
];
visual.Card.Opacity = exiting && animation is OverlayAnimation.Fade or OverlayAnimation.FadeSlide or OverlayAnimation.ScaleFade ? 0 : 1;
visual.Translate.X = exiting && animation is OverlayAnimation.Slide or OverlayAnimation.FadeSlide ? -settings.SlideDistance : 0;
var scale = exiting && animation == OverlayAnimation.ScaleFade ? settings.ScaleAmount : 1;
visual.Scale.ScaleX = visual.Scale.ScaleY = scale;
}
private static Easing EasingFor(string value) => value switch
{
"linear" => new LinearEasing(),
"cubic-in-out" => new CubicEaseInOut(),
_ => new CubicEaseOut()
};
private Control CreateChatContent(OverlayFeedMessage message, OverlayContainerSettings settings)
{
var grid = new Grid { ColumnDefinitions = ColumnDefinitions.Parse(settings.ShowAvatar && message.Author.Avatar is not null ? "40,*" : "*"), ColumnSpacing = 10 };
var contentColumn = 0;
if (settings.ShowAvatar && message.Author.Avatar is not null)
{
var avatar = new AnimatedOverlayImage { Width = 36, Height = 36, Stretch = Stretch.UniformToFill };
_ = LoadImageAsync(avatar, message.Author.Avatar);
grid.Children.Add(avatar);
contentColumn = 1;
}
var stack = new StackPanel { Spacing = 3 };
var header = new StackPanel { Orientation = Avalonia.Layout.Orientation.Horizontal, Spacing = 7 };
if (settings.ShowPlatform) header.Children.Add(Text(message.Platform.ToUpperInvariant(), settings, 11, FontWeight.Bold));
foreach (var badge in message.Author.Badges.Take(6))
{
if (badge.Image is null)
{
header.Children.Add(Text(badge.Label, settings, 10, FontWeight.SemiBold));
continue;
}
var token = new Grid();
ToolTip.SetTip(token, badge.Label);
var fallback = Text(badge.Label, settings, 10, FontWeight.SemiBold);
var image = new AnimatedOverlayImage { Width = 18, Height = 18, Stretch = Stretch.Uniform, Opacity = 0 };
token.Children.Add(fallback);
token.Children.Add(image);
header.Children.Add(token);
_ = LoadImageAsync(image, badge.Image, fallback);
}
if (settings.ShowUsername) header.Children.Add(Text(message.Author.Name, settings, settings.FontSize * .72, FontWeight.Bold));
if (header.Children.Count > 0) stack.Children.Add(header);
stack.Children.Add(CreateMessageBody(message, settings));
var media = CreateMessageMedia(message);
if (media is not null) stack.Children.Add(media);
Grid.SetColumn(stack, contentColumn);
grid.Children.Add(stack);
return grid;
}
private Control CreateMessageBody(OverlayFeedMessage message, OverlayContainerSettings settings)
{
var panel = new WrapPanel { Orientation = Avalonia.Layout.Orientation.Horizontal };
var cursor = 0;
foreach (var emote in message.Emotes.OrderBy(value => value.Start))
{
if (emote.Start < cursor || emote.Start > message.Text.Length) continue;
if (emote.Start > cursor) panel.Children.Add(Text(message.Text[cursor..emote.Start], settings, settings.FontSize));
if (emote.Image is not null)
{
var token = new Grid { Margin = new Thickness(2, 0) };
var fallback = Text(string.IsNullOrWhiteSpace(emote.Label) ? message.Text[emote.Start..Math.Min(message.Text.Length, emote.End + 1)] : emote.Label, settings, settings.FontSize);
var image = new AnimatedOverlayImage { Width = settings.FontSize * 1.3, Height = settings.FontSize * 1.3, Opacity = 0 };
token.Children.Add(fallback);
token.Children.Add(image);
panel.Children.Add(token);
_ = LoadImageAsync(image, emote.Image, fallback);
}
else panel.Children.Add(Text(emote.Label, settings, settings.FontSize));
cursor = Math.Min(message.Text.Length, emote.End + 1);
}
if (cursor < message.Text.Length) panel.Children.Add(Text(message.Text[cursor..], settings, settings.FontSize));
if (panel.Children.Count == 0) panel.Children.Add(Text(message.Text, settings, settings.FontSize));
return panel;
}
private Control? CreateMessageMedia(OverlayFeedMessage message)
{
var panel = new WrapPanel
{
Orientation = Avalonia.Layout.Orientation.Horizontal,
ItemWidth = 220,
ItemHeight = 150
};
foreach (var media in message.Media.Take(4))
{
var source = media.Type == "video" ? media.Preview : media.Url;
if (string.IsNullOrWhiteSpace(source)) continue;
var image = new AnimatedOverlayImage
{
Width = 220,
Height = 150,
Stretch = Stretch.Uniform
};
if (!string.IsNullOrWhiteSpace(media.Alt)) ToolTip.SetTip(image, media.Alt);
panel.Children.Add(image);
_ = LoadImageAsync(image, source);
}
return panel.Children.Count == 0 ? null : panel;
}
private Control CreateEventContent(OverlayRenderedEvent value, OverlayContainerSettings settings)
{
var stack = new StackPanel { Spacing = 3 };
if (settings.ShowPlatform) stack.Children.Add(Text(value.Platform.ToUpperInvariant(), settings, 11, FontWeight.Bold));
stack.Children.Add(Text(value.Summary, settings, settings.FontSize, FontWeight.SemiBold));
return stack;
}
private static TextBlock Text(string value, OverlayContainerSettings settings, double size, FontWeight? weight = null)
{
var block = new TextBlock
{
Text = value,
TextWrapping = TextWrapping.Wrap,
Foreground = Brush.Parse(settings.TextColor),
TextAlignment = settings.HorizontalAlignment switch
{
OverlayHorizontalAlignment.Left => TextAlignment.Left,
OverlayHorizontalAlignment.Middle => TextAlignment.Center,
_ => TextAlignment.Right
},
FontFamily = new FontFamily(settings.FontFamily),
FontSize = size,
FontWeight = weight ?? FontWeight.Normal
};
block.Effect = new DropShadowEffect
{
Color = OverlayColor.ColorFromRgba(settings.TextShadow),
OffsetX = settings.ShadowX,
OffsetY = settings.ShadowY,
BlurRadius = settings.ShadowBlur
};
return block;
}
private async Task LoadImageAsync(AnimatedOverlayImage target, string url, Control? fallback = null)
{
var asset = await _images.GetAsync(url).ConfigureAwait(false);
await Dispatcher.UIThread.InvokeAsync(() =>
{
if (asset is not null)
{
target.SetAsset(asset);
target.Opacity = 1;
if (fallback is not null) fallback.IsVisible = false;
}
});
}
private Screen? ResolveScreen()
{
var all = Screens.All;
var primary = Screens.Primary ?? all.FirstOrDefault();
var resolution = OverlayMonitorPolicy.Resolve(_settings.MonitorId, all.Select(ScreenId), primary is null ? null : ScreenId(primary));
var selected = all.FirstOrDefault(screen => ScreenId(screen) == resolution.MonitorId);
if (resolution.Warning != _temporaryMonitorWarning)
{
_temporaryMonitorWarning = resolution.Warning;
NativeStatusChanged?.Invoke();
}
return selected;
}
public IReadOnlyList<(string Id, string Label)> Monitors() => Screens.All.Select((screen, index) =>
(ScreenId(screen), $"{(screen.IsPrimary ? "Primary · " : "")}{screen.DisplayName ?? $"Monitor {index + 1}"} · {screen.Bounds.Width}×{screen.Bounds.Height}")).ToList();
public static string ScreenId(Screen screen) => !string.IsNullOrWhiteSpace(screen.DisplayName)
? $"display:{screen.DisplayName}"
: $"bounds:{screen.Bounds.X},{screen.Bounds.Y},{screen.Bounds.Width},{screen.Bounds.Height}";
private static PhysicalRect Logical(PhysicalRect value, double scaling) => new(
(int)Math.Round(value.X / scaling),
(int)Math.Round(value.Y / scaling),
Math.Max(1, (int)Math.Round(value.Width / scaling)),
Math.Max(1, (int)Math.Round(value.Height / scaling)));
private void ApplyMonitorAndStyles()
{
if (!Dispatcher.UIThread.CheckAccess()) { Dispatcher.UIThread.Post(ApplyMonitorAndStyles); return; }
var screen = ResolveScreen();
if (screen is null) return;
Position = screen.Bounds.Position;
Width = screen.Bounds.Width / screen.Scaling;
Height = screen.Bounds.Height / screen.Scaling;
var hwnd = TryGetPlatformHandle()?.Handle ?? IntPtr.Zero;
if (hwnd == IntPtr.Zero) return;
var style = GetWindowLongPtr(hwnd, GwlExStyle).ToInt64() | WsExToolWindow | WsExLayered;
if (_editing) style &= ~(WsExTransparent | WsExNoActivate);
else style |= WsExTransparent | WsExNoActivate;
SetWindowLongPtr(hwnd, GwlExStyle, new IntPtr(style));
SetWindowPos(hwnd, HwndTopmost, screen.Bounds.X, screen.Bounds.Y, screen.Bounds.Width, screen.Bounds.Height, SwpNoActivate | SwpFrameChanged);
if (SetWindowDisplayAffinity(hwnd, WdaExcludeFromCapture))
CaptureExclusion = new(CaptureExclusionState.Active, "Windows capture exclusion is active (best effort; capture software may still include the overlay).");
else
{
var error = Marshal.GetLastWin32Error();
CaptureExclusion = error == 87
? new(CaptureExclusionState.Unsupported, "This Windows/capture path does not support display-affinity exclusion.")
: new(CaptureExclusionState.Failed, $"Windows could not apply capture exclusion (error {error}).");
}
NativeStatusChanged?.Invoke();
if (!_visibleRequested && IsVisible) Hide();
}
private void OnScreensChanged(object? sender, EventArgs args) => Dispatcher.UIThread.Post(ApplyMonitorAndStyles);
private void OnSystemDisplayChanged(object? sender, EventArgs args) => Dispatcher.UIThread.Post(ApplyMonitorAndStyles);
private void OnPowerChanged(object sender, PowerModeChangedEventArgs args) => Dispatcher.UIThread.Post(ApplyMonitorAndStyles);
private void OnSessionSwitch(object sender, SessionSwitchEventArgs args) => Dispatcher.UIThread.Post(ApplyMonitorAndStyles);
public ValueTask DisposeAsync()
{
if (_disposed) return ValueTask.CompletedTask;
_disposed = true;
_timer.Stop();
Screens.Changed -= OnScreensChanged;
SystemEvents.DisplaySettingsChanged -= OnSystemDisplayChanged;
SystemEvents.PowerModeChanged -= OnPowerChanged;
SystemEvents.SessionSwitch -= OnSessionSwitch;
_images.Dispose();
Close();
return ValueTask.CompletedTask;
}
[DllImport("user32.dll", EntryPoint = "GetWindowLongPtrW", SetLastError = true)] private static extern IntPtr GetWindowLongPtr(IntPtr hwnd, int index);
[DllImport("user32.dll", EntryPoint = "SetWindowLongPtrW", SetLastError = true)] private static extern IntPtr SetWindowLongPtr(IntPtr hwnd, int index, IntPtr value);
[DllImport("user32.dll", SetLastError = true)] private static extern bool SetWindowPos(IntPtr hwnd, IntPtr after, int x, int y, int cx, int cy, uint flags);
[DllImport("user32.dll", SetLastError = true)] private static extern bool SetWindowDisplayAffinity(IntPtr hwnd, uint affinity);
private sealed record CardVisual(string Id, Border Card, TranslateTransform Translate, ScaleTransform Scale);
}
internal sealed class HttpsImageCache : IDisposable
{
private const int Limit = 128;
private const int MaxEncodedBytes = 2 * 1024 * 1024;
private const int MaxDimension = 384;
private const int MaxFrames = 60;
private const long MaxDecodedPixels = 12_000_000;
private readonly HttpClient _http = new() { Timeout = TimeSpan.FromSeconds(5) };
private readonly ConcurrentDictionary<string, Lazy<Task<OverlayImageAsset?>>> _entries = new();
private readonly Queue<string> _order = new();
private readonly object _gate = new();
public Task<OverlayImageAsset?> GetAsync(string value)
{
if (!Uri.TryCreate(value, UriKind.Absolute, out var uri) || uri.Scheme != Uri.UriSchemeHttps) return Task.FromResult<OverlayImageAsset?>(null);
lock (_gate)
{
if (!_entries.ContainsKey(uri.AbsoluteUri))
{
while (_entries.Count >= Limit && _order.TryDequeue(out var oldest))
if (_entries.TryRemove(oldest, out var removed) && removed.IsValueCreated && removed.Value.IsCompletedSuccessfully) removed.Value.Result?.Dispose();
_order.Enqueue(uri.AbsoluteUri);
}
}
return _entries.GetOrAdd(uri.AbsoluteUri, key => new(() => DownloadAsync(key))).Value;
}
private async Task<OverlayImageAsset?> DownloadAsync(string url)
{
try
{
using var response = await _http.GetAsync(url, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);
if (!response.IsSuccessStatusCode || response.Content.Headers.ContentLength > MaxEncodedBytes) return null;
var bytes = await response.Content.ReadAsByteArrayAsync().ConfigureAwait(false);
if (bytes.Length > MaxEncodedBytes) return null;
return Decode(bytes);
}
catch { return null; }
}
private static OverlayImageAsset? Decode(byte[] bytes)
{
using var data = SKData.CreateCopy(bytes);
using var codec = SKCodec.Create(data);
if (codec is null || codec.Info.Width <= 0 || codec.Info.Height <= 0) return null;
var scale = Math.Min(1d, MaxDimension / (double)Math.Max(codec.Info.Width, codec.Info.Height));
var dimensions = codec.GetScaledDimensions((float)scale);
var width = Math.Max(1, dimensions.Width);
var height = Math.Max(1, dimensions.Height);
var frameCount = Math.Clamp(codec.FrameCount, 1, MaxFrames);
frameCount = (int)Math.Min(frameCount, Math.Max(1, MaxDecodedPixels / ((long)width * height)));
var info = new SKImageInfo(width, height, SKColorType.Bgra8888, SKAlphaType.Premul);
using var working = new SKBitmap(info);
var frameInfo = codec.FrameInfo;
var frames = new List<Bitmap>(frameCount);
var delays = new List<TimeSpan>(frameCount);
try
{
for (var index = 0; index < frameCount; index++)
{
var requiredFrame = index < frameInfo.Length ? frameInfo[index].RequiredFrame : -1;
if (requiredFrame < 0) working.Erase(SKColors.Transparent);
var result = codec.GetPixels(info, working.GetPixels(), working.RowBytes, new SKCodecOptions(index, requiredFrame));
if (result is not (SKCodecResult.Success or SKCodecResult.IncompleteInput)) break;
using var image = SKImage.FromBitmap(working);
using var encoded = image.Encode(SKEncodedImageFormat.Png, 100);
frames.Add(new Bitmap(new MemoryStream(encoded.ToArray())));
var duration = index < frameInfo.Length ? frameInfo[index].Duration : 100;
delays.Add(TimeSpan.FromMilliseconds(Math.Clamp(duration <= 0 ? 100 : duration, 20, 10_000)));
}
return frames.Count == 0 ? null : new OverlayImageAsset(frames, delays);
}
catch
{
foreach (var frame in frames) frame.Dispose();
return null;
}
}
public void Dispose()
{
_http.Dispose();
foreach (var value in _entries.Values)
if (value.IsValueCreated && value.Value.IsCompletedSuccessfully) value.Value.Result?.Dispose();
}
}
internal sealed class OverlayImageAsset(IReadOnlyList<Bitmap> frames, IReadOnlyList<TimeSpan> delays) : IDisposable
{
public IReadOnlyList<Bitmap> Frames { get; } = frames;
public IReadOnlyList<TimeSpan> Delays { get; } = delays;
public void Dispose()
{
foreach (var frame in Frames) frame.Dispose();
}
}
internal sealed class AnimatedOverlayImage : Image
{
private DispatcherTimer? _timer;
private OverlayImageAsset? _asset;
private int _frame;
public AnimatedOverlayImage()
{
DetachedFromVisualTree += (_, _) => Stop();
AttachedToVisualTree += (_, _) => Start();
}
public void SetAsset(OverlayImageAsset asset)
{
Stop();
_asset = asset;
_frame = 0;
Source = asset.Frames[0];
Start();
}
private void Start()
{
if (_asset is null || _asset.Frames.Count < 2 || _timer is not null) return;
_timer = new DispatcherTimer { Interval = DelayFor(0) };
_timer.Tick += Advance;
_timer.Start();
}
private void Advance(object? sender, EventArgs args)
{
if (_asset is null || _asset.Frames.Count < 2) return;
_frame = (_frame + 1) % _asset.Frames.Count;
Source = _asset.Frames[_frame];
if (_timer is not null) _timer.Interval = DelayFor(_frame);
}
private TimeSpan DelayFor(int frame) =>
_asset is not null && frame < _asset.Delays.Count ? _asset.Delays[frame] : TimeSpan.FromMilliseconds(100);
private void Stop()
{
if (_timer is null) return;
_timer.Stop();
_timer.Tick -= Advance;
_timer = null;
}
}
internal static class OverlayColor
{
public static Color ColorFromRgba(string value)
{
var hex = value.TrimStart('#');
if (hex.Length != 8) return Color.Parse(value);
return Color.FromArgb(
Convert.ToByte(hex[6..8], 16),
Convert.ToByte(hex[0..2], 16),
Convert.ToByte(hex[2..4], 16),
Convert.ToByte(hex[4..6], 16));
}
public static IBrush BrushFromRgba(string value) => new SolidColorBrush(ColorFromRgba(value));
}

View File

@ -0,0 +1,48 @@
using System.Text.Json;
using System.Text.Json.Serialization;
namespace Lumi.Companion.Overlay;
public sealed class OverlayFeedAuthor
{
[JsonPropertyName("name")] public string Name { get; set; } = "";
[JsonPropertyName("username")] public string Username { get; set; } = "";
[JsonPropertyName("avatar")] public string? Avatar { get; set; }
[JsonPropertyName("badges")] public List<OverlayFeedBadge> Badges { get; set; } = [];
}
public sealed class OverlayFeedBadge
{
[JsonPropertyName("label")] public string Label { get; set; } = "";
[JsonPropertyName("image")] public string? Image { get; set; }
}
public sealed class OverlayFeedEmote
{
[JsonPropertyName("start")] public int Start { get; set; }
[JsonPropertyName("end")] public int End { get; set; }
[JsonPropertyName("label")] public string Label { get; set; } = "";
[JsonPropertyName("image")] public string? Image { get; set; }
}
public sealed class OverlayFeedMedia
{
[JsonPropertyName("url")] public string? Url { get; set; }
[JsonPropertyName("type")] public string Type { get; set; } = "image";
[JsonPropertyName("alt")] public string Alt { get; set; } = "";
[JsonPropertyName("preview")] public string? Preview { get; set; }
}
public sealed class OverlayFeedMessage
{
[JsonPropertyName("id")] public string Id { get; set; } = "";
[JsonPropertyName("platform")] public string Platform { get; set; } = "";
[JsonPropertyName("text")] public string Text { get; set; } = "";
[JsonPropertyName("author")] public OverlayFeedAuthor Author { get; set; } = new();
[JsonPropertyName("emotes")] public List<OverlayFeedEmote> Emotes { get; set; } = [];
[JsonPropertyName("media")] public List<OverlayFeedMedia> Media { get; set; } = [];
}
public sealed record OverlayRenderedEvent(string Id, string Type, string Platform, string Summary, JsonElement Payload);
public sealed record OverlayFeedStatus(string Platform, string State, string Detail);

View File

@ -0,0 +1,90 @@
namespace Lumi.Companion.Overlay;
public enum OverlayItemState { Queued, Entering, Visible, Exiting, Removed }
public sealed class OverlayQueueItem(string id, string kind, object payload, DateTimeOffset queuedAt)
{
public string Id { get; } = id;
public string Kind { get; } = kind;
public object Payload { get; } = payload;
public DateTimeOffset QueuedAt { get; } = queuedAt;
public DateTimeOffset StateAt { get; internal set; } = queuedAt;
public OverlayItemState State { get; internal set; } = OverlayItemState.Queued;
}
public sealed class OverlayLifecycleQueue(Func<OverlayContainerSettings> settings, Func<DateTimeOffset>? now = null)
{
private readonly List<OverlayQueueItem> _items = [];
private readonly Func<DateTimeOffset> _now = now ?? (() => DateTimeOffset.UtcNow);
public IReadOnlyList<OverlayQueueItem> Items => _items;
public event Action? Changed;
public bool Add(string id, string kind, object payload)
{
if (_items.Any(item => item.Id == id && item.State != OverlayItemState.Removed)) return false;
var value = settings();
if (_items.Count(item => item.State != OverlayItemState.Removed) >= Math.Max(value.QueueLimit + 1, value.QueueLimit * 2))
return false;
var active = _items.Where(item => item.State is not OverlayItemState.Exiting and not OverlayItemState.Removed).ToList();
while (active.Count >= value.QueueLimit)
{
var oldest = active[0];
oldest.State = OverlayItemState.Exiting;
oldest.StateAt = _now();
active.RemoveAt(0);
}
var created = new OverlayQueueItem(id, kind, payload, _now()) { State = OverlayItemState.Entering, StateAt = _now() };
_items.Add(created);
Changed?.Invoke();
return true;
}
public void Advance()
{
var current = _now();
var value = settings();
var changed = false;
foreach (var item in _items)
{
var elapsed = current - item.StateAt;
if (item.State == OverlayItemState.Entering && elapsed.TotalMilliseconds >= value.EntryDurationMs)
{
item.State = OverlayItemState.Visible; item.StateAt = current; changed = true;
}
else if (item.State == OverlayItemState.Visible && elapsed.TotalMilliseconds >= value.LifetimeMs)
{
item.State = OverlayItemState.Exiting; item.StateAt = current; changed = true;
}
else if (item.State == OverlayItemState.Exiting && elapsed.TotalMilliseconds >= value.ExitDurationMs)
{
item.State = OverlayItemState.Removed; changed = true;
}
}
if (_items.RemoveAll(item => item.State == OverlayItemState.Removed) > 0) changed = true;
if (changed) Changed?.Invoke();
}
public void Clear()
{
var current = _now();
foreach (var item in _items.Where(item => item.State is not OverlayItemState.Exiting and not OverlayItemState.Removed))
{
item.State = OverlayItemState.Exiting;
item.StateAt = current;
}
Changed?.Invoke();
}
public void ExitWhere(Func<OverlayQueueItem, bool> predicate)
{
var current = _now();
var changed = false;
foreach (var item in _items.Where(item => item.State is not OverlayItemState.Exiting and not OverlayItemState.Removed && predicate(item)))
{
item.State = OverlayItemState.Exiting;
item.StateAt = current;
changed = true;
}
if (changed) Changed?.Invoke();
}
}

View File

@ -0,0 +1,72 @@
namespace Lumi.Companion.Overlay;
public static class OverlayVisibilityPolicy
{
public static bool Automatic(LumiOverlaySettings settings, IEnumerable<OverlaySource> sources)
{
if (!settings.Enabled || settings.Visibility == OverlayVisibilityMode.Off) return false;
if (settings.Visibility == OverlayVisibilityMode.On) return true;
var selected = settings.SelectedSourceIds.ToHashSet(StringComparer.Ordinal);
return sources.Any(source => source.Live == true &&
source.Platform is "twitch" or "youtube" &&
selected.Contains(source.Id));
}
public static bool Effective(LumiOverlaySettings settings, IEnumerable<OverlaySource> sources, bool preview, bool? visibilityOverride) =>
preview || settings.Enabled && (visibilityOverride ?? Automatic(settings, sources));
}
public sealed record OverlayVisibilityPresentation(bool Visible, string State, string Action, bool CanToggle);
public static class OverlayVisibilityPresenter
{
public static OverlayVisibilityPresentation Describe(
LumiOverlaySettings settings,
bool automaticVisible,
bool preview,
bool? visibilityOverride)
{
if (preview)
return new(true, "Shown — sample preview is controlling visibility", "End preview to change visibility", false);
if (!settings.Enabled)
return new(false, "Hidden — Lumi Overlay is disabled", "Enable and save Lumi Overlay first", false);
var visible = visibilityOverride ?? automaticVisible;
if (visibilityOverride.HasValue)
return new(visible,
visible ? "Shown — temporary override" : "Hidden — temporary override",
"Restore automatic visibility",
true);
var automaticDetail = settings.Visibility switch
{
OverlayVisibilityMode.Off => "visibility is set to Off",
OverlayVisibilityMode.WhenLive when visible => "a selected channel is live",
OverlayVisibilityMode.WhenLive => "waiting for a selected channel to go live",
_ => "visibility is set to Always on"
};
return new(visible,
$"{(visible ? "Shown" : "Hidden")} — automatic; {automaticDetail}",
visible ? "Hide temporarily" : "Show temporarily",
true);
}
}
public sealed record MonitorResolution(string? MonitorId, bool UsedFallback, string? Warning);
public static class OverlayMonitorPolicy
{
public static MonitorResolution Resolve(string? savedMonitorId, IEnumerable<string> availableMonitorIds, string? primaryMonitorId)
{
var available = availableMonitorIds.Where(value => !string.IsNullOrWhiteSpace(value)).ToHashSet(StringComparer.Ordinal);
if (savedMonitorId is not null && available.Contains(savedMonitorId)) return new(savedMonitorId, false, null);
var fallback = primaryMonitorId is not null && available.Contains(primaryMonitorId) ? primaryMonitorId : available.FirstOrDefault();
return new(fallback, savedMonitorId is not null,
savedMonitorId is null ? null : "The selected monitor is unavailable. Lumi Overlay is temporarily using the primary monitor.");
}
}
public static class CompanionLiveReconnectPolicy
{
public static TimeSpan Delay(int attempt) => TimeSpan.FromSeconds(Math.Min(30, Math.Pow(2, Math.Min(Math.Max(1, attempt), 5))));
}

View File

@ -0,0 +1,192 @@
using System.Text.Json.Serialization;
namespace Lumi.Companion.Overlay;
public enum OverlayVisibilityMode { Off, WhenLive, On }
public enum OverlayLayoutMode { Unified, DividedHorizontal, DividedVertical, Separate }
public enum OverlayAnchor { TopLeft, Top, TopRight, Left, Right, BottomLeft, Bottom, BottomRight }
public enum OverlayHorizontalAlignment { Left, Middle, Right }
public enum OverlayVerticalAlignment { Top, Middle, Bottom }
public enum OverlayNewestDirection { Start, End }
public enum OverlayAnimation { None, Fade, Slide, FadeSlide, ScaleFade }
public sealed class OverlaySourceSetting
{
public string Id { get; set; } = "";
public string Platform { get; set; } = "";
public string Label { get; set; } = "";
public bool Selected { get; set; }
}
public sealed class OverlayContainerSettings
{
public OverlayAnchor Anchor { get; set; } = OverlayAnchor.BottomLeft;
public int PaddingTop { get; set; } = 24;
public int PaddingRight { get; set; } = 24;
public int PaddingBottom { get; set; } = 24;
public int PaddingLeft { get; set; } = 24;
public int Width { get; set; } = 700;
public int Height { get; set; } = 600;
public OverlayHorizontalAlignment HorizontalAlignment { get; set; } = OverlayHorizontalAlignment.Left;
public OverlayVerticalAlignment VerticalAlignment { get; set; } = OverlayVerticalAlignment.Bottom;
public OverlayNewestDirection NewestDirection { get; set; } = OverlayNewestDirection.End;
public string CardBackground { get; set; } = "#12121266";
public string TextColor { get; set; } = "#bbbbbb";
public string FontFamily { get; set; } = "Segoe UI";
public double FontSize { get; set; } = 22;
public string TextShadow { get; set; } = "#000000aa";
public int ShadowX { get; set; } = 1;
public int ShadowY { get; set; } = 2;
public int ShadowBlur { get; set; } = 3;
public bool ShowAvatar { get; set; } = true;
public bool ShowUsername { get; set; } = true;
public bool ShowPlatform { get; set; } = true;
public OverlayAnimation EntryAnimation { get; set; } = OverlayAnimation.FadeSlide;
public OverlayAnimation ExitAnimation { get; set; } = OverlayAnimation.FadeSlide;
public int EntryDurationMs { get; set; } = 260;
public int ExitDurationMs { get; set; } = 220;
public string Easing { get; set; } = "cubic-out";
public int SlideDistance { get; set; } = 24;
public double ScaleAmount { get; set; } = 0.92;
public int LifetimeMs { get; set; } = 12000;
public int QueueLimit { get; set; } = 12;
public OverlayContainerSettings CreateCopy() => new()
{
Anchor = Anchor,
PaddingTop = PaddingTop,
PaddingRight = PaddingRight,
PaddingBottom = PaddingBottom,
PaddingLeft = PaddingLeft,
Width = Width,
Height = Height,
HorizontalAlignment = HorizontalAlignment,
VerticalAlignment = VerticalAlignment,
NewestDirection = NewestDirection,
CardBackground = CardBackground,
TextColor = TextColor,
FontFamily = FontFamily,
FontSize = FontSize,
TextShadow = TextShadow,
ShadowX = ShadowX,
ShadowY = ShadowY,
ShadowBlur = ShadowBlur,
ShowAvatar = ShowAvatar,
ShowUsername = ShowUsername,
ShowPlatform = ShowPlatform,
EntryAnimation = EntryAnimation,
ExitAnimation = ExitAnimation,
EntryDurationMs = EntryDurationMs,
ExitDurationMs = ExitDurationMs,
Easing = Easing,
SlideDistance = SlideDistance,
ScaleAmount = ScaleAmount,
LifetimeMs = LifetimeMs,
QueueLimit = QueueLimit
};
}
public sealed class LumiOverlaySettings
{
public int SchemaVersion { get; set; } = 1;
public string ActiveProfileId { get; set; } = "default";
public bool Enabled { get; set; } = true;
public OverlayVisibilityMode Visibility { get; set; } = OverlayVisibilityMode.On;
public string? MonitorId { get; set; }
public List<OverlaySourceSetting> Sources { get; set; } = [];
public HashSet<string> EventTypes { get; set; } = [];
public OverlayLayoutMode Layout { get; set; } = OverlayLayoutMode.Unified;
public int ChatPercentage { get; set; } = 70;
public OverlayContainerSettings Unified { get; set; } = new();
public OverlayContainerSettings Chat { get; set; } = new();
public OverlayContainerSettings Events { get; set; } = new() { Anchor = OverlayAnchor.TopRight, Width = 520, Height = 420 };
public string ToggleVisibilityHotkey { get; set; } = "Ctrl+Alt+L";
public string TogglePreviewHotkey { get; set; } = "Ctrl+Alt+P";
[JsonIgnore]
public IEnumerable<string> SelectedSourceIds => Sources.Where(source => source.Selected).Select(source => source.Id);
public LumiOverlaySettings CreateCopy() => new()
{
SchemaVersion = SchemaVersion,
ActiveProfileId = ActiveProfileId,
Enabled = Enabled,
Visibility = Visibility,
MonitorId = MonitorId,
Sources = Sources.Select(source => new OverlaySourceSetting
{
Id = source.Id,
Platform = source.Platform,
Label = source.Label,
Selected = source.Selected
}).ToList(),
EventTypes = EventTypes.ToHashSet(StringComparer.Ordinal),
Layout = Layout,
ChatPercentage = ChatPercentage,
Unified = Unified.CreateCopy(),
Chat = Chat.CreateCopy(),
Events = Events.CreateCopy(),
ToggleVisibilityHotkey = ToggleVisibilityHotkey,
TogglePreviewHotkey = TogglePreviewHotkey
};
public void Normalize()
{
SchemaVersion = 1;
ActiveProfileId = Clean(ActiveProfileId, 80, "default");
MonitorId = string.IsNullOrWhiteSpace(MonitorId) ? null : MonitorId.Trim()[..Math.Min(MonitorId.Trim().Length, 300)];
Sources = Sources.Where(source => !string.IsNullOrWhiteSpace(source.Id))
.GroupBy(source => source.Id, StringComparer.Ordinal).Select(group => group.First()).Take(250).ToList();
foreach (var source in Sources)
{
source.Id = Clean(source.Id, 300, "");
source.Platform = Clean(source.Platform, 24, "");
source.Label = Clean(source.Label, 300, source.Id);
}
EventTypes = EventTypes.Where(value => !string.IsNullOrWhiteSpace(value)).Select(value => Clean(value, 120, "")).Take(100).ToHashSet();
ChatPercentage = Math.Clamp(ChatPercentage, 10, 90);
NormalizeContainer(Unified);
NormalizeContainer(Chat);
NormalizeContainer(Events);
ToggleVisibilityHotkey = Clean(ToggleVisibilityHotkey, 80, "Ctrl+Alt+L");
TogglePreviewHotkey = Clean(TogglePreviewHotkey, 80, "Ctrl+Alt+P");
}
private static void NormalizeContainer(OverlayContainerSettings value)
{
value.Width = Math.Clamp(value.Width, 160, 16000);
value.Height = Math.Clamp(value.Height, 100, 16000);
value.PaddingTop = Math.Clamp(value.PaddingTop, 0, 4000);
value.PaddingRight = Math.Clamp(value.PaddingRight, 0, 4000);
value.PaddingBottom = Math.Clamp(value.PaddingBottom, 0, 4000);
value.PaddingLeft = Math.Clamp(value.PaddingLeft, 0, 4000);
AnchorLayout.ClampPadding(value);
value.CardBackground = ColorValue(value.CardBackground, true, "#12121266");
value.TextColor = ColorValue(value.TextColor, false, "#bbbbbb");
value.TextShadow = ColorValue(value.TextShadow, true, "#000000aa");
value.FontFamily = Clean(value.FontFamily, 120, "Segoe UI");
value.FontSize = Math.Clamp(value.FontSize, 8, 120);
value.ShadowX = Math.Clamp(value.ShadowX, -40, 40);
value.ShadowY = Math.Clamp(value.ShadowY, -40, 40);
value.ShadowBlur = Math.Clamp(value.ShadowBlur, 0, 40);
value.EntryDurationMs = Math.Clamp(value.EntryDurationMs, 0, 5000);
value.ExitDurationMs = Math.Clamp(value.ExitDurationMs, 0, 5000);
value.SlideDistance = Math.Clamp(value.SlideDistance, 0, 1000);
value.ScaleAmount = Math.Clamp(value.ScaleAmount, 0.1, 2);
value.LifetimeMs = Math.Clamp(value.LifetimeMs, 1000, 300000);
value.QueueLimit = Math.Clamp(value.QueueLimit, 1, 100);
value.Easing = Clean(value.Easing, 40, "cubic-out");
}
private static string Clean(string? value, int max, string fallback)
{
var clean = (value ?? "").Trim();
return clean.Length == 0 ? fallback : clean[..Math.Min(clean.Length, max)];
}
private static string ColorValue(string? value, bool alpha, string fallback) =>
System.Text.RegularExpressions.Regex.IsMatch(value ?? "", alpha ? "^#[0-9a-fA-F]{8}$" : "^#[0-9a-fA-F]{6}$") ? value! : fallback;
}
public sealed record OverlaySource(string Id, string Platform, string Label, bool? Live, bool Available, string Status);
public sealed record OverlayEventType(string Id, string Label, string Platform);

View File

@ -0,0 +1,40 @@
using System.Text.Json;
namespace Lumi.Companion.Overlay;
public sealed class OverlaySettingsStore(string path)
{
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web) { WriteIndented = true };
public LumiOverlaySettings Current { get; private set; } = new();
public async Task LoadAsync(CancellationToken cancellationToken = default)
{
if (!File.Exists(path)) { Current.Normalize(); return; }
try
{
await using var stream = File.OpenRead(path);
Current = await JsonSerializer.DeserializeAsync<LumiOverlaySettings>(stream, JsonOptions, cancellationToken).ConfigureAwait(false) ?? new();
}
catch (JsonException)
{
var backup = path + $".invalid-{DateTimeOffset.UtcNow:yyyyMMddHHmmss}";
File.Copy(path, backup, true);
Current = new();
}
Current.Normalize();
}
public async Task SaveAsync(LumiOverlaySettings value, CancellationToken cancellationToken = default)
{
value.Normalize();
Directory.CreateDirectory(Path.GetDirectoryName(path)!);
var temporary = path + ".tmp";
await using (var stream = new FileStream(temporary, FileMode.Create, FileAccess.Write, FileShare.None))
{
await JsonSerializer.SerializeAsync(stream, value, JsonOptions, cancellationToken).ConfigureAwait(false);
await stream.FlushAsync(cancellationToken).ConfigureAwait(false);
}
File.Move(temporary, path, true);
Current = value;
}
}

View File

@ -0,0 +1,16 @@
{
"id": "lumi_overlay",
"name": "Lumi Overlay",
"version": "0.1.2",
"provider_api": 1,
"capabilities": [
"network.lumi.overlay.read",
"windows.monitor.overlay",
"windows.capture-exclusion",
"windows.global-hotkeys"
],
"navigation": {
"root": "Lumi Overlay",
"pages": ["LumiOverlay"]
}
}

View File

@ -0,0 +1,66 @@
namespace Lumi.Companion.SongOverlay;
public sealed record MediaSourceDefinition(string Id, string Label, string Detail, bool EnabledByDefault);
public static class MediaSourceCatalog
{
public static IReadOnlyList<MediaSourceDefinition> All { get; } =
[
new("wmp", "Windows Media Player", "Windows Media Player and the modern Windows Media Player app", true),
new("youtube", "YouTube / YouTube Music", "YouTube playback exposed by Chrome or Edge", true),
new("spotify", "Spotify", "Spotify desktop playback", true),
new("vlc", "VLC", "VLC media playback", false),
new("apple_music", "iTunes / Apple Music", "iTunes and Apple Music for Windows", false),
new("soundcloud", "SoundCloud", "SoundCloud playback exposed by a browser or app", false),
new("tidal", "TIDAL", "TIDAL playback exposed by a browser or app", false),
new("bandcamp", "Bandcamp", "Bandcamp playback exposed by a browser", false),
new("qobuz", "Qobuz", "Qobuz playback exposed by a browser or app", false)
];
public static HashSet<string> Defaults() =>
All.Where(source => source.EnabledByDefault).Select(source => source.Id).ToHashSet(StringComparer.Ordinal);
public static string? Detect(string? sourceAppId, string? title, string? artist, string? album)
{
var source = (sourceAppId ?? "").ToLowerInvariant();
var metadata = string.Join(" ", title, artist, album).ToLowerInvariant();
if (source.Contains("spotify")) return "spotify";
if (source.Contains("vlc")) return "vlc";
if (source.Contains("itunes") || source.Contains("applemusic") || source.Contains("apple music")) return "apple_music";
if (source.Contains("soundcloud")) return "soundcloud";
if (source.Contains("tidal")) return "tidal";
if (source.Contains("bandcamp")) return "bandcamp";
if (source.Contains("qobuz")) return "qobuz";
if (source.Contains("wmplayer") || source.Contains("zunemusic") || source.Contains("mediaplayer")) return "wmp";
var browser = source.Contains("chrome") || source.Contains("msedge") || source.Contains("firefox");
if (!browser) return null;
if (metadata.Contains("soundcloud")) return "soundcloud";
if (metadata.Contains("tidal")) return "tidal";
if (metadata.Contains("bandcamp")) return "bandcamp";
if (metadata.Contains("qobuz")) return "qobuz";
if (metadata.Contains("youtube") || metadata.Contains("yt music")) return "youtube";
// Browser media sessions do not consistently expose their origin. Treat
// an otherwise unidentified browser session as YouTube, the enabled
// browser default, while named services remain independently switchable.
return "youtube";
}
public static string? DetectEnabled(
string? sourceAppId,
string? title,
string? artist,
string? album,
IReadOnlySet<string> enabled)
{
var detected = Detect(sourceAppId, title, artist, album);
if (detected != "youtube" || enabled.Contains("youtube")) return detected;
var source = (sourceAppId ?? "").ToLowerInvariant();
if (!source.Contains("chrome") && !source.Contains("msedge") && !source.Contains("firefox")) return detected;
return new[] { "soundcloud", "tidal", "bandcamp", "qobuz" }.FirstOrDefault(enabled.Contains);
}
public static string Label(string? id) =>
All.FirstOrDefault(source => source.Id == id)?.Label ?? "Windows media";
}

View File

@ -7,29 +7,30 @@ using Windows.Storage.Streams;
namespace Lumi.Companion.SongOverlay.Providers;
internal sealed class SpotifyWindowsMediaProvider : IMediaProvider
internal sealed class WindowsMediaProvider : IMediaProvider
{
private const int MaxCoverInputBytes = 3 * 1024 * 1024;
private const int MaxCoverOutputBytes = 500 * 1024;
private readonly Func<SongOverlaySettings> _settings;
private readonly Func<MediaTrack, CancellationToken, Task<MediaTrack>> _enrich;
private readonly Action<string, Exception?> _log;
private readonly Action<string, string, Exception?> _log;
private GlobalSystemMediaTransportControlsSessionManager? _manager;
private GlobalSystemMediaTransportControlsSession? _session;
private MediaSnapshot? _last;
private MediaTrack? _trackCache;
private string _sourceId = "windows-media";
private bool _started;
private readonly SemaphoreSlim _refreshLock = new(1, 1);
public SpotifyWindowsMediaProvider(Func<SongOverlaySettings> settings, Func<MediaTrack, CancellationToken, Task<MediaTrack>> enrich, Action<string, Exception?> log)
public WindowsMediaProvider(Func<SongOverlaySettings> settings, Func<MediaTrack, CancellationToken, Task<MediaTrack>> enrich, Action<string, string, Exception?> log)
{
_settings = settings;
_enrich = enrich;
_log = log;
}
public string Id => "spotify";
public string DisplayName => "Spotify";
public string Id => "windows-media";
public string DisplayName => "Windows media discovery";
public event EventHandler<ProviderStateChangedEventArgs>? StateChanged;
public event EventHandler<string>? AvailabilityChanged;
@ -62,22 +63,22 @@ internal sealed class SpotifyWindowsMediaProvider : IMediaProvider
private async void OnSessionCollectionChanged(GlobalSystemMediaTransportControlsSessionManager sender, object args)
{
try { await SelectSessionAsync(CancellationToken.None).ConfigureAwait(false); }
catch (Exception error) { _log("Could not refresh Spotify media sessions", error); }
catch (Exception error) { _log("media_session_refresh_failed", "Could not refresh Windows media sessions", error); }
}
private async Task SelectSessionAsync(CancellationToken cancellationToken)
{
var selected = _manager?.GetSessions().FirstOrDefault(IsSpotifySession);
var selected = await SelectEnabledSessionAsync(cancellationToken).ConfigureAwait(false);
if (ReferenceEquals(selected, _session))
{
if (_session is not null) await RefreshAndPublishAsync("snapshot", false, cancellationToken).ConfigureAwait(false);
if (_session is not null) await RefreshAndPublishAsync("media", true, cancellationToken).ConfigureAwait(false);
return;
}
DetachSession();
_session = selected;
if (_session is null)
{
AvailabilityChanged?.Invoke(this, "Spotify is not exposing a Windows media session.");
AvailabilityChanged?.Invoke(this, "No enabled source is exposing a Windows media session.");
if (_last is not null)
{
_last = null;
@ -88,7 +89,7 @@ internal sealed class SpotifyWindowsMediaProvider : IMediaProvider
_session.MediaPropertiesChanged += OnMediaPropertiesChanged;
_session.PlaybackInfoChanged += OnPlaybackInfoChanged;
_session.TimelinePropertiesChanged += OnTimelinePropertiesChanged;
AvailabilityChanged?.Invoke(this, "Spotify media session connected.");
AvailabilityChanged?.Invoke(this, $"{MediaSourceCatalog.Label(_sourceId)} media session connected.");
await RefreshAndPublishAsync("snapshot", true, cancellationToken).ConfigureAwait(false);
}
@ -104,20 +105,20 @@ internal sealed class SpotifyWindowsMediaProvider : IMediaProvider
private async void OnMediaPropertiesChanged(GlobalSystemMediaTransportControlsSession sender, MediaPropertiesChangedEventArgs args)
{
try { await RefreshAndPublishAsync("media", true, CancellationToken.None).ConfigureAwait(false); }
catch (Exception error) { _log("Spotify media-property update failed", error); }
try { await SelectSessionAsync(CancellationToken.None).ConfigureAwait(false); }
catch (Exception error) { _log("media_property_update_failed", "Windows media-property update failed", error); }
}
private async void OnPlaybackInfoChanged(GlobalSystemMediaTransportControlsSession sender, PlaybackInfoChangedEventArgs args)
{
try { await RefreshAndPublishAsync("playback", false, CancellationToken.None).ConfigureAwait(false); }
catch (Exception error) { _log("Spotify playback update failed", error); }
catch (Exception error) { _log("media_playback_update_failed", "Windows media playback update failed", error); }
}
private async void OnTimelinePropertiesChanged(GlobalSystemMediaTransportControlsSession sender, TimelinePropertiesChangedEventArgs args)
{
try { await RefreshAndPublishAsync("timeline", false, CancellationToken.None).ConfigureAwait(false); }
catch (Exception error) { _log("Spotify timeline update failed", error); }
catch (Exception error) { _log("media_timeline_update_failed", "Windows media timeline update failed", error); }
}
private async Task RefreshAndPublishAsync(string reason, bool enrichTrack, CancellationToken cancellationToken)
@ -162,16 +163,18 @@ internal sealed class SpotifyWindowsMediaProvider : IMediaProvider
var title = Clean(media.Title);
var artist = Clean(media.Artist);
var album = Clean(media.AlbumTitle);
_sourceId = MediaSourceCatalog.DetectEnabled(
_session.SourceAppUserModelId, title, artist, album, _settings().EnabledSources) ?? _sourceId;
if (!string.IsNullOrWhiteSpace(title) || !string.IsNullOrWhiteSpace(artist))
{
var key = Fingerprint(title, artist, album, duration);
var cover = _settings().SendCoverArt ? await ReadCoverAsync(media.Thumbnail, cancellationToken).ConfigureAwait(false) : null;
var fallbackLink = _settings().UseSearchLinkFallback
var fallbackLink = _sourceId == "spotify" && _settings().UseSearchLinkFallback
? "https://open.spotify.com/search/" + Uri.EscapeDataString(string.Join(" ", new[] { title, artist }.Where(value => value.Length > 0)))
: "";
var track = new MediaTrack(key, title, artist, album, "", fallbackLink, duration, cover);
_trackCache = track;
if (enrichTrack) _ = EnrichAndPublishAsync(track);
if (enrichTrack && _sourceId == "spotify") _ = EnrichAndPublishAsync(track);
}
else
{
@ -183,7 +186,7 @@ internal sealed class SpotifyWindowsMediaProvider : IMediaProvider
_trackCache = _trackCache with { DurationMilliseconds = duration };
}
var snapshot = new MediaSnapshot(Id, status, position, duration, rate, _trackCache, capturedAt);
var snapshot = new MediaSnapshot(_sourceId, status, position, duration, rate, _trackCache, capturedAt);
_last = snapshot;
return snapshot;
}
@ -206,13 +209,37 @@ internal sealed class SpotifyWindowsMediaProvider : IMediaProvider
finally { _refreshLock.Release(); }
await RefreshAndPublishAsync("metadata", false, CancellationToken.None).ConfigureAwait(false);
}
catch (Exception error) { _log("Spotify metadata enrichment update failed", error); }
catch (Exception error) { _log("spotify_metadata_update_failed", "Spotify metadata enrichment update failed", error); }
}
private static bool IsSpotifySession(GlobalSystemMediaTransportControlsSession session)
private async Task<GlobalSystemMediaTransportControlsSession?> SelectEnabledSessionAsync(CancellationToken cancellationToken)
{
var source = session.SourceAppUserModelId ?? "";
return source.Contains("spotify", StringComparison.OrdinalIgnoreCase);
if (_manager is null) return null;
GlobalSystemMediaTransportControlsSession? fallback = null;
string? fallbackSource = null;
foreach (var session in _manager.GetSessions())
{
cancellationToken.ThrowIfCancellationRequested();
try
{
var media = await session.TryGetMediaPropertiesAsync();
var sourceId = MediaSourceCatalog.DetectEnabled(
session.SourceAppUserModelId, media.Title, media.Artist, media.AlbumTitle, _settings().EnabledSources);
if (sourceId is null || !_settings().EnabledSources.Contains(sourceId)) continue;
var playing = session.GetPlaybackInfo().PlaybackStatus ==
GlobalSystemMediaTransportControlsSessionPlaybackStatus.Playing;
if (playing)
{
_sourceId = sourceId;
return session;
}
fallback ??= session;
fallbackSource ??= sourceId;
}
catch { }
}
if (fallbackSource is not null) _sourceId = fallbackSource;
return fallback;
}
private static async Task<CoverPayload?> ReadCoverAsync(IRandomAccessStreamReference? reference, CancellationToken cancellationToken)

View File

@ -1,6 +1,7 @@
using Lumi.Companion.Abstractions;
using Lumi.Companion.SongOverlay.Providers;
using Lumi.Companion.SongOverlay.Spotify;
using System.Text.Json;
namespace Lumi.Companion.SongOverlay;
@ -44,7 +45,7 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
public CompanionPluginDescriptor Descriptor { get; } = new(
PluginId,
"Song Overlay",
new Version(0, 1, 1),
new Version(0, 1, 2),
"Reads provider-neutral Windows media-session events and sends minimal playback changes to Lumi.",
200);
@ -67,7 +68,7 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
public async Task InitializeAsync(CancellationToken cancellationToken = default)
{
await _settingsStore.LoadAsync(cancellationToken).ConfigureAwait(false);
if (string.IsNullOrWhiteSpace(Settings.ProviderId)) Settings.ProviderId = "spotify";
Settings.Normalize();
// Re-save through the current schema so legacy plugin-specific Lumi host
// and connection-key fields are removed after the shared transport upgrade.
await _settingsStore.SaveAsync(Settings, cancellationToken).ConfigureAwait(false);
@ -78,9 +79,7 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
public async Task SaveSettingsAsync(bool restartProvider, CancellationToken cancellationToken = default)
{
Settings.HeartbeatSeconds = Math.Clamp(Settings.HeartbeatSeconds, 15, 300);
Settings.SeekThresholdMilliseconds = Math.Clamp(Settings.SeekThresholdMilliseconds, 500, 10000);
Settings.ProviderId = string.IsNullOrWhiteSpace(Settings.ProviderId) ? "spotify" : Settings.ProviderId.Trim().ToLowerInvariant();
Settings.Normalize();
await _settingsStore.SaveAsync(Settings, cancellationToken).ConfigureAwait(false);
if (restartProvider) await RestartProviderAsync(cancellationToken).ConfigureAwait(false);
RaiseChanged();
@ -114,7 +113,7 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
if (!_transport.IsConfigured) throw new InvalidOperationException("Pair Companion with Lumi before sending song updates.");
if (_provider is null) throw new InvalidOperationException("No media provider is running.");
var snapshot = await _provider.GetSnapshotAsync(cancellationToken).ConfigureAwait(false);
if (snapshot is null) throw new InvalidOperationException("Spotify is not currently exposing playback information.");
if (snapshot is null) throw new InvalidOperationException("None of the enabled media sources is currently exposing playback information.");
await SendAsync("snapshot", snapshot, includeTrack: true, cancellationToken).ConfigureAwait(false);
}
@ -129,11 +128,7 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
_lastObserved = null;
_lastDelivered = null;
_lifetime = new CancellationTokenSource();
_provider = Settings.ProviderId switch
{
"spotify" => new SpotifyWindowsMediaProvider(() => Settings, EnrichTrackAsync, Log),
_ => throw new InvalidOperationException($"Provider '{Settings.ProviderId}' is not installed.")
};
_provider = new WindowsMediaProvider(() => Settings, EnrichTrackAsync, Log);
_provider.StateChanged += OnProviderStateChanged;
_provider.AvailabilityChanged += OnAvailabilityChanged;
try
@ -142,15 +137,15 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
StartHeartbeat();
var initial = await _provider.GetSnapshotAsync(cancellationToken).ConfigureAwait(false);
if (!_transport.IsConfigured)
SetStatus(CompanionPluginHealth.Warning, "Pairing required", "Spotify monitoring is ready, but Companion must be paired before Lumi can receive updates.");
SetStatus(CompanionPluginHealth.Warning, "Pairing required", "Media monitoring is ready, but Companion must be paired before Lumi can receive updates.");
else if (initial is null)
SetStatus(CompanionPluginHealth.Warning, "Waiting for Spotify", "Spotify is not currently exposing a Windows media session.");
SetStatus(CompanionPluginHealth.Warning, "Waiting for media", "No enabled source is currently exposing a Windows media session.");
else
SetStatus(CompanionPluginHealth.Healthy, "Monitoring", Describe(initial));
}
catch (Exception error)
{
SetStatus(CompanionPluginHealth.Error, "Provider failed", "Could not start the Spotify media provider: " + error.Message);
SetStatus(CompanionPluginHealth.Error, "Provider failed", "Could not start Windows media discovery: " + error.Message);
throw;
}
}
@ -192,7 +187,7 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
}
}
catch (OperationCanceledException) { }
catch (Exception error) { Log("Song Overlay heartbeat failed", error); }
catch (Exception error) { Log("heartbeat_failed", "Song Overlay heartbeat failed", error); }
}, token);
}
@ -203,9 +198,10 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
SetStatus(CompanionPluginHealth.Warning, "Pairing required", message);
return;
}
var waiting = message.Contains("not exposing", StringComparison.OrdinalIgnoreCase);
var waiting = message.Contains("not exposing", StringComparison.OrdinalIgnoreCase) ||
message.Contains("No enabled source", StringComparison.OrdinalIgnoreCase);
SetStatus(waiting ? CompanionPluginHealth.Warning : CompanionPluginHealth.Healthy,
waiting ? "Waiting for Spotify" : "Monitoring", message);
waiting ? "Waiting for media" : "Monitoring", message);
}
private void OnProviderStateChanged(object? sender, ProviderStateChangedEventArgs args) =>
@ -274,7 +270,7 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
catch (OperationCanceledException) { }
catch (Exception error)
{
Log("Could not process a Song Overlay event", error);
Log("event_processing_failed", "Could not process a Song Overlay event", error);
SetStatus(CompanionPluginHealth.Warning, "Delivery delayed", "Playback was detected, but Lumi could not be updated: " + error.Message);
}
finally { _eventLock.Release(); }
@ -317,7 +313,7 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
if (enriched is null) return track;
CoverPayload? cover = track.Cover;
if (enriched.CoverBytes is { Length: > 0 } bytes && Settings.SendCoverArt)
cover = SpotifyWindowsMediaProvider.CreateCoverPayload(bytes, enriched.CoverMime) ?? cover;
cover = WindowsMediaProvider.CreateCoverPayload(bytes, enriched.CoverMime) ?? cover;
return track with
{
Link = string.IsNullOrWhiteSpace(enriched.Link) ? track.Link : enriched.Link,
@ -358,17 +354,49 @@ public sealed class SongOverlayRuntime : ICompanionPluginContribution, IAsyncDis
}
private void RaiseChanged() => Changed?.Invoke();
private void Log(string message, Exception? error = null)
private void Log(string eventId, string message, Exception? error = null)
{
try
{
Directory.CreateDirectory(_logDirectory);
File.AppendAllText(Path.Combine(_logDirectory, $"song-overlay-{DateTime.UtcNow:yyyyMMdd}.log"),
$"{DateTimeOffset.Now:O}\t{message}{(error is null ? "" : "\t" + error)}{Environment.NewLine}");
PruneLogs();
var normalizedEvent = CompanionLogSanitizer.NormalizeEvent(eventId);
var line = JsonSerializer.Serialize(new
{
timestamp = DateTimeOffset.UtcNow,
level = CompanionLogSanitizer.LevelForEvent(normalizedEvent),
source = $"plugin:{PluginId}",
category = "plugin",
@event = normalizedEvent,
message = CompanionLogSanitizer.Sanitize(message),
error = error is null ? null : CompanionLogSanitizer.Sanitize(error.ToString())
});
File.AppendAllText(
Path.Combine(_logDirectory, $"song-overlay-{DateTime.UtcNow:yyyy-MM-dd}.jsonl"),
line + Environment.NewLine
);
}
catch { }
}
private void PruneLogs()
{
var files = new DirectoryInfo(_logDirectory)
.GetFiles("song-overlay-*.*")
.Where(file => file.Extension is ".jsonl" or ".log")
.OrderBy(file => file.CreationTimeUtc)
.ToList();
foreach (var file in files.Where(file => file.CreationTimeUtc < DateTime.UtcNow.AddDays(-7))) file.Delete();
const long cap = 32L * 1024 * 1024;
var total = files.Where(file => file.Exists).Sum(file => file.Length);
foreach (var file in files.Where(file => file.Exists))
{
if (total <= cap) break;
total -= file.Length;
file.Delete();
}
}
private static string Describe(MediaSnapshot snapshot) => snapshot.Track is null
? "Connected; no active song."
: $"{snapshot.Status}: {snapshot.Track.Title} — {snapshot.Track.Artist}";

View File

@ -4,10 +4,22 @@ public sealed class SongOverlaySettings
{
public bool Enabled { get; set; } = true;
public string ProviderId { get; set; } = "spotify";
public HashSet<string> EnabledSources { get; set; } = MediaSourceCatalog.Defaults();
public int HeartbeatSeconds { get; set; } = 30;
public int SeekThresholdMilliseconds { get; set; } = 1500;
public bool SendCoverArt { get; set; } = true;
public bool UseSearchLinkFallback { get; set; } = true;
public string SpotifyClientId { get; set; } = "";
public string ProtectedSpotifyRefreshToken { get; set; } = "";
public void Normalize()
{
ProviderId = "windows-media";
EnabledSources ??= MediaSourceCatalog.Defaults();
EnabledSources = EnabledSources
.Where(id => MediaSourceCatalog.All.Any(source => source.Id == id))
.ToHashSet(StringComparer.Ordinal);
HeartbeatSeconds = Math.Clamp(HeartbeatSeconds, 15, 300);
SeekThresholdMilliseconds = Math.Clamp(SeekThresholdMilliseconds, 500, 10000);
}
}

View File

@ -16,12 +16,12 @@ internal sealed class SpotifyWebApiEnricher : IDisposable
private readonly HttpClient _http = new() { Timeout = TimeSpan.FromSeconds(15) };
private readonly SongOverlaySecretProtector _secrets;
private readonly Action _save;
private readonly Action<string, Exception?> _log;
private readonly Action<string, string, Exception?> _log;
private readonly SongOverlaySettings _settings;
private string _accessToken = "";
private DateTimeOffset _accessTokenExpiresAt = DateTimeOffset.MinValue;
public SpotifyWebApiEnricher(SongOverlaySettings settings, SongOverlaySecretProtector secrets, Action save, Action<string, Exception?> log)
public SpotifyWebApiEnricher(SongOverlaySettings settings, SongOverlaySecretProtector secrets, Action save, Action<string, string, Exception?> log)
{
_settings = settings;
_secrets = secrets;
@ -130,7 +130,7 @@ internal sealed class SpotifyWebApiEnricher : IDisposable
}
catch (Exception error)
{
_log("Spotify metadata enrichment failed", error);
_log("spotify_metadata_enrichment_failed", "Spotify metadata enrichment failed", error);
return null;
}
}

View File

@ -3,7 +3,7 @@
"name": "Song Overlay",
"version": "0.1.2",
"provider_api": 1,
"providers": ["spotify"],
"providers": ["windows-media", "spotify", "wmp", "youtube", "vlc", "apple-music", "soundcloud", "tidal", "bandcamp", "qobuz"],
"capabilities": [
"windows.media-session.read",
"network.lumi",

View File

@ -1,5 +1,6 @@
using System.Buffers.Binary;
using System.IO.Pipes;
using System.Collections.Concurrent;
using System.Text.Json;
using Lumi.Companion.Protocol;
@ -7,12 +8,13 @@ namespace Lumi.Companion.Transcription;
public sealed class ObsBridgePipe : IAsyncDisposable
{
private static readonly HashSet<string> AllowedTypes = ["hello", "source_list", "source_state", "selection_state", "obs_state", "health"];
private static readonly HashSet<string> AllowedTypes = ["hello", "source_list", "source_state", "selection_state", "obs_state", "health", "stream_test_service_state", "stream_test_metrics"];
private readonly string _pipeName;
private readonly Func<JsonElement, Task> _onMessage;
private readonly Func<ReadOnlyMemory<byte>, Task> _onAudio;
private readonly CancellationTokenSource _lifetime = new();
private readonly SemaphoreSlim _writeLock = new(1, 1);
private readonly ConcurrentDictionary<string, TaskCompletionSource<JsonElement>> _requests = new();
private Task? _loop;
private Stream? _connection;
@ -55,7 +57,10 @@ public sealed class ObsBridgePipe : IAsyncDisposable
using var json = JsonDocument.Parse(message);
var type = json.RootElement.TryGetProperty("type", out var property) ? property.GetString() : null;
if (type is null || !AllowedTypes.Contains(type)) throw new InvalidDataException("OBS bridge message type is not allowed.");
await _onMessage(json.RootElement.Clone());
var clone = json.RootElement.Clone();
if (clone.TryGetProperty("request_id", out var requestId) && requestId.ValueKind == JsonValueKind.String &&
_requests.TryRemove(requestId.GetString()!, out var pending)) pending.TrySetResult(clone);
await _onMessage(clone);
}
}
public static async Task WriteAsync(Stream stream, object message, CancellationToken cancellationToken)
@ -79,6 +84,25 @@ public sealed class ObsBridgePipe : IAsyncDisposable
catch (IOException) { return false; }
finally { _writeLock.Release(); }
}
public async Task<JsonElement> RequestAsync(string type, object payload, TimeSpan timeout, CancellationToken cancellationToken = default)
{
var requestId = Guid.NewGuid().ToString();
var completion = new TaskCompletionSource<JsonElement>(TaskCreationOptions.RunContinuationsAsynchronously);
if (!_requests.TryAdd(requestId, completion)) throw new InvalidOperationException("OBS request identity could not be reserved.");
using var timeoutSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeoutSource.CancelAfter(timeout);
using var registration = timeoutSource.Token.Register(() => completion.TrySetCanceled(timeoutSource.Token));
try
{
if (!await SendAsync(new { type, request_id = requestId, payload }, timeoutSource.Token))
throw new InvalidOperationException("OBS is not connected to Lumi Companion.");
return await completion.Task;
}
finally
{
_requests.TryRemove(requestId, out _);
}
}
private static async Task ReadExactlyAsync(Stream stream, Memory<byte> buffer, CancellationToken cancellationToken)
{
var read = 0;
@ -89,5 +113,5 @@ public sealed class ObsBridgePipe : IAsyncDisposable
read += count;
}
}
public async ValueTask DisposeAsync() { _lifetime.Cancel(); if (_loop is not null) try { await _loop; } catch (OperationCanceledException) { } _writeLock.Dispose(); _lifetime.Dispose(); }
public async ValueTask DisposeAsync() { _lifetime.Cancel(); foreach (var request in _requests.Values) request.TrySetCanceled(); _requests.Clear(); if (_loop is not null) try { await _loop; } catch (OperationCanceledException) { } _writeLock.Dispose(); _lifetime.Dispose(); }
}

View File

@ -1,6 +1,6 @@
param(
[string]$ObsVersion = "31.1.1",
[string]$BridgeVersion = "0.1.0",
[string]$BridgeVersion = "0.2.5",
[string]$CacheRoot = "$env:LOCALAPPDATA\LumiCompanionBuild"
)
@ -17,6 +17,12 @@ $sourceRoot = Join-Path $sourceParent "obs-studio-$ObsVersion"
$importRoot = Join-Path $obsRoot "imports"
$buildRoot = Join-Path $obsRoot "bridge-build"
function Invoke-Native([string]$FilePath, [string[]]$Arguments) {
$argumentLine = ($Arguments | ForEach-Object { '"' + ([string]$_).Replace('"', '\"') + '"' }) -join ' '
$process = Start-Process -FilePath $FilePath -ArgumentList $argumentLine -NoNewWindow -Wait -PassThru
return $process.ExitCode
}
$runtimeUrl = "https://github.com/obsproject/obs-studio/releases/download/$ObsVersion/OBS-Studio-$ObsVersion-Windows-x64.zip"
$sourceUrl = "https://github.com/obsproject/obs-studio/archive/refs/tags/$ObsVersion.zip"
$runtimeSha256 = "9d8dceb77acd8af04af23f877061f63c9bef78ca73d2093d0ccba1bb9104173f"
@ -46,8 +52,8 @@ function New-ImportLibrary([string]$Dll, [string]$Name, [string]$Dumpbin, [strin
if ($_ -match '^\s+\d+\s+[0-9A-F]+\s+[0-9A-F]+\s+(\S+)') { $Matches[1] }
})
@("LIBRARY $Name", "EXPORTS") + ($exports | ForEach-Object { " $_" }) | Set-Content -Encoding Ascii $definition
& $LibExe /nologo /machine:x64 "/def:$definition" "/out:$(Join-Path $importRoot "$Name.lib")"
if ($LASTEXITCODE) { throw "Could not create the $Name import library." }
$exitCode = Invoke-Native $LibExe @("/nologo", "/machine:x64", "/def:$definition", "/out:$(Join-Path $importRoot "$Name.lib")")
if ($exitCode) { throw "Could not create the $Name import library." }
}
Get-VerifiedArchive $runtimeUrl $runtimeArchive $runtimeSha256
@ -88,10 +94,8 @@ $configureArguments = @(
"-DOBS_IMPORT_DIR=$importRoot",
"-DLUMI_BRIDGE_VERSION=$BridgeVersion"
)
& $cmake @configureArguments
if ($LASTEXITCODE) { throw "OBS bridge configuration failed." }
& $cmake --build $buildRoot --config Release
if ($LASTEXITCODE) { throw "OBS bridge build failed." }
if ((Invoke-Native $cmake $configureArguments)) { throw "OBS bridge configuration failed." }
if ((Invoke-Native $cmake @("--build", $buildRoot, "--config", "Release"))) { throw "OBS bridge build failed." }
New-Item -ItemType Directory -Force -Path $componentRoot | Out-Null
$bridgeDll = Join-Path $buildRoot "Release\lumi-obs-bridge.dll"

View File

@ -1,6 +1,6 @@
param(
[string]$Version = "0.1.0",
[string]$BridgeVersion = "0.1.0",
[string]$Version = "0.2.8",
[string]$BridgeVersion = "0.2.5",
[string]$ObsVersion = "31.1.1"
)
@ -16,6 +16,12 @@ $installer = Join-Path $outputRoot "Lumi.Companion-Setup.exe"
$legalSourceRoot = Join-Path $repoRoot "companion\legal"
$publishLegalRoot = Join-Path $publishRoot "legal"
function Invoke-Native([string]$FilePath, [string[]]$Arguments) {
$argumentLine = ($Arguments | ForEach-Object { '"' + ([string]$_).Replace('"', '\"') + '"' }) -join ' '
$process = Start-Process -FilePath $FilePath -ArgumentList $argumentLine -NoNewWindow -Wait -PassThru
return $process.ExitCode
}
function Compress-ArchiveWithRetry {
param(
[Parameter(Mandatory = $true)][string]$Path,
@ -43,13 +49,11 @@ $localDotnet = Join-Path $HOME ".dotnet-sdk\dotnet.exe"
$dotnet = if (Test-Path $localDotnet) { $localDotnet } else { (Get-Command dotnet.exe -ErrorAction Stop).Source }
$publishArguments = @("publish", $project, "-c", "Release", "-r", "win-x64", "--self-contained", "true", "--disable-build-servers", "-o", $publishRoot,
"-p:PublishSingleFile=true", "-p:IncludeNativeLibrariesForSelfExtract=true", "-p:DebugType=None", "-p:Version=$Version")
& $dotnet @publishArguments
if ($LASTEXITCODE) { throw "Companion publish failed." }
if ((Invoke-Native $dotnet $publishArguments)) { throw "Companion publish failed." }
$bridgeDiagnostic = Join-Path $outputRoot "obs-bridge-package-diagnostic.json"
Remove-Item $bridgeDiagnostic -Force -ErrorAction SilentlyContinue
& (Join-Path $publishRoot "Lumi.Companion.App.exe") --diagnose-obs-bridge $bridgeDiagnostic
if ($LASTEXITCODE -ne 0) {
if ((Invoke-Native (Join-Path $publishRoot "Lumi.Companion.App.exe") @("--diagnose-obs-bridge", $bridgeDiagnostic))) {
$detail = if (Test-Path $bridgeDiagnostic) { Get-Content $bridgeDiagnostic -Raw } else { "No diagnostic result was written." }
throw "Published OBS integration payload failed its executable-level check: $detail"
}
@ -123,8 +127,7 @@ $compileArguments = @(
"/DOutputRoot=$outputRoot",
(Join-Path $repoRoot "companion\installer\Lumi.Companion.iss")
)
& $iscc @compileArguments
if ($LASTEXITCODE -ne 0 -or -not (Test-Path $installer)) { throw "Companion installer compilation failed." }
if ((Invoke-Native $iscc $compileArguments) -or -not (Test-Path $installer)) { throw "Companion installer compilation failed." }
$file = Get-Item $archive
$sha = (Get-FileHash $archive -Algorithm SHA256).Hash.ToLowerInvariant()

View File

@ -0,0 +1,20 @@
$ErrorActionPreference = "Stop"
$repoRoot = (Resolve-Path (Join-Path $PSScriptRoot "..\..")).Path
$sdk = Join-Path $env:LOCALAPPDATA "Lumi\dotnet-sdk-10\dotnet.exe"
$dotnet = if (Test-Path $sdk) { $sdk } else { (Get-Command dotnet.exe -ErrorAction Stop).Source }
$node = (Get-Command node.exe -ErrorAction Stop).Source
Push-Location $repoRoot
try {
& $node "plugins/lumi_overlay/tests/verify.js"
if ($LASTEXITCODE) { throw "Lumi Overlay server verification failed." }
& $node "plugins/lumi_transcription/tests/verify.js"
if ($LASTEXITCODE) { throw "Companion capability verification failed." }
& $dotnet run --project "companion/tests/Lumi.Companion.Core.Tests/Lumi.Companion.Core.Tests.csproj" --configuration Release
if ($LASTEXITCODE) { throw "Lumi Overlay Companion verification failed." }
& $dotnet build "companion/Lumi.Companion.sln" --configuration Release
if ($LASTEXITCODE) { throw "Lumi Companion build failed." }
Write-Host "Lumi Overlay verification passed." -ForegroundColor Green
}
finally {
Pop-Location
}

View File

@ -1,12 +1,17 @@
$ErrorActionPreference = 'Stop'
$repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..')).Path
$sdk = Join-Path $env:LOCALAPPDATA "Lumi\dotnet-sdk-10\dotnet.exe"
$dotnet = if (Test-Path $sdk) { $sdk } else { (Get-Command dotnet.exe -ErrorAction Stop).Source }
$node = (Get-Command node.exe -ErrorAction Stop).Source
Push-Location $repoRoot
try {
Write-Host 'Building Lumi Companion and all registered Companion plugins...'
dotnet build 'companion/Lumi.Companion.sln' -c Release -p:EnableWindowsTargeting=true
& $dotnet build 'companion/Lumi.Companion.sln' -c Release
if ($LASTEXITCODE) { throw 'Lumi Companion build failed.' }
Write-Host 'Verifying the existing Lumi Song Overlay server plugin...'
node 'plugins/now_playing/tests/verify.js'
& $node 'plugins/now_playing/tests/verify.js'
if ($LASTEXITCODE) { throw 'Song Overlay server verification failed.' }
Write-Host 'Song Overlay Companion verification passed.' -ForegroundColor Green
}

View File

@ -0,0 +1,53 @@
using System.Text.RegularExpressions;
namespace Lumi.Companion.Abstractions;
public static partial class CompanionLogSanitizer
{
public static string Sanitize(string? value, int maxLength = 4000)
{
var safeMaxLength = Math.Clamp(maxLength, 64, 65536);
var text = value ?? "";
text = AuthorizationPattern().Replace(text, "$1 [redacted]");
text = CookiePattern().Replace(text, "$1[redacted]");
text = QuerySecretPattern().Replace(text, "$1[redacted]");
text = AssignedSecretPattern().Replace(text, "$1[redacted]");
return text.Length <= safeMaxLength ? text : text[..safeMaxLength] + "\n[truncated]";
}
public static string NormalizeEvent(string? value)
{
var normalized = InvalidEventCharacters().Replace((value ?? "").Trim().ToLowerInvariant(), "_");
normalized = RepeatedUnderscores().Replace(normalized, "_").Trim('_');
return string.IsNullOrWhiteSpace(normalized) ? "unknown_event" : normalized[..Math.Min(80, normalized.Length)];
}
public static string LevelForEvent(string eventId)
{
var normalized = NormalizeEvent(eventId);
if (normalized.EndsWith("_failed", StringComparison.Ordinal) ||
normalized.EndsWith("_error", StringComparison.Ordinal))
{
return "error";
}
return normalized is "disconnected" or "audio_dropped" ? "warn" : "info";
}
[GeneratedRegex(@"\b(Bearer|Basic|LumiDevice)\s+[A-Za-z0-9._~+/=-]+", RegexOptions.IgnoreCase)]
private static partial Regex AuthorizationPattern();
[GeneratedRegex(@"\b((?:set-)?cookie\s*:\s*)[^\r\n]+", RegexOptions.IgnoreCase)]
private static partial Regex CookiePattern();
[GeneratedRegex(@"([?&](?:token|key|secret|password|authorization)=)[^&#\s]+", RegexOptions.IgnoreCase)]
private static partial Regex QuerySecretPattern();
[GeneratedRegex(@"\b((?:api[_-]?key|access[_-]?token|refresh[_-]?token|client[_-]?secret|password|passwd|secret|signature|authorization)\s*[:=]\s*)[^\s,;}]+", RegexOptions.IgnoreCase)]
private static partial Regex AssignedSecretPattern();
[GeneratedRegex(@"[^a-z0-9]+")]
private static partial Regex InvalidEventCharacters();
[GeneratedRegex(@"_+")]
private static partial Regex RepeatedUnderscores();
}

View File

@ -7,6 +7,30 @@ public sealed record CompanionPluginHttpResponse(
public bool IsSuccessStatusCode => StatusCode is >= 200 and <= 299;
}
public enum CompanionPluginLiveState
{
Connecting,
Connected,
Reconnecting,
WaitingForPairing,
Stopped
}
public sealed record CompanionPluginLiveStatus(
CompanionPluginLiveState State,
string Detail,
int Attempt = 0);
/// <summary>
/// An authenticated, plugin-scoped live channel. It intentionally exposes no
/// credential or authorization header.
/// </summary>
public interface ICompanionPluginLiveChannel
{
Task SendJsonAsync(object payload, CancellationToken cancellationToken = default);
IAsyncEnumerable<string> ReadAllAsync(CancellationToken cancellationToken = default);
}
/// <summary>
/// A plugin-scoped view of the Companion shell's authenticated Lumi transport.
/// Plugins never receive, store, or refresh the paired device credential.
@ -19,4 +43,14 @@ public interface ICompanionPluginTransport
string relativePath,
object payload,
CancellationToken cancellationToken = default);
/// <summary>
/// Runs a live plugin connection. The Companion shell owns authentication,
/// TLS policy, reconnect backoff and socket lifetime.
/// </summary>
Task RunLiveJsonAsync(
string relativePath,
Func<ICompanionPluginLiveChannel, CancellationToken, Task> connected,
Action<CompanionPluginLiveStatus>? status,
CancellationToken cancellationToken = default);
}

View File

@ -32,10 +32,16 @@ public sealed record CompanionPluginAction(
Func<bool>? CanExecute = null,
int Order = 100);
public static class CompanionPluginNavigation
{
public static bool UsesNestedNavigation(IReadOnlyCollection<CompanionPluginPage> pages) => pages.Count > 1;
}
/// <summary>
/// A UI-neutral contribution contract. The Avalonia shell owns rendering and
/// guarantees that each plugin receives one nested root in both the sidebar and
/// tray menu instead of leaking loose actions into the global navigation.
/// gives a single-page plugin one direct sidebar entry. Plugins with multiple
/// pages receive one collapsible sidebar root instead of leaking loose pages
/// into the global navigation.
/// </summary>
public interface ICompanionPluginContribution
{

View File

@ -73,10 +73,6 @@
<Setter Property="CornerRadius" Value="9" />
<Setter Property="Padding" Value="16,9" />
</Style>
<Style Selector="Expander.pluginRoot">
<Setter Property="HorizontalAlignment" Value="Stretch" />
<Setter Property="Margin" Value="0,1" />
</Style>
<Style Selector="Button.nav">
<Setter Property="HorizontalContentAlignment" Value="Left" />
<Setter Property="Background" Value="Transparent" />
@ -85,6 +81,9 @@
<Setter Property="Padding" Value="13,10" />
<Setter Property="Margin" Value="0,2" />
</Style>
<Style Selector="Button.navRoot">
<Setter Property="HorizontalContentAlignment" Value="Stretch" />
</Style>
<Style Selector="Button.nav.selected">
<Setter Property="Background" Value="#DDEEF0" />
<Setter Property="Foreground" Value="{DynamicResource LumiPrimaryBrush}" />

View File

@ -5,6 +5,7 @@ using Avalonia.Markup.Xaml;
using Avalonia.Threading;
using Lumi.Companion.Abstractions;
using Lumi.Companion.SongOverlay;
using Lumi.Companion.Overlay;
namespace Lumi.Companion.App;
@ -13,6 +14,7 @@ public partial class App : Application
private TrayIcon? _trayIcon;
private MainWindow? _mainWindow;
private SongOverlayRuntime? _songOverlay;
private LumiOverlayRuntime? _lumiOverlay;
private IReadOnlyList<ICompanionPluginContribution> _plugins = [];
public override void Initialize() => AvaloniaXamlLoader.Load(this);
@ -29,8 +31,11 @@ public partial class App : Application
Path.Combine(paths.Root, "plugins", "now_playing"),
paths.LogsDirectory,
runtime.CreatePluginTransport("now_playing"));
_plugins = [new TranscriptionPluginContribution(runtime), _songOverlay];
_mainWindow = new MainWindow(runtime, settings, _songOverlay, _plugins);
_lumiOverlay = new LumiOverlayRuntime(
Path.Combine(paths.Root, "plugins", "lumi_overlay"),
runtime.CreatePluginTransport("lumi_overlay"));
_plugins = [new TranscriptionPluginContribution(runtime), _lumiOverlay, _songOverlay];
_mainWindow = new MainWindow(runtime, settings, _songOverlay, _lumiOverlay, _plugins);
desktop.MainWindow = _mainWindow;
ConfigureTray(desktop, runtime, _plugins);
runtime.UpdateRestartRequested += async () =>
@ -38,21 +43,24 @@ public partial class App : Application
_mainWindow.AllowExit();
_trayIcon?.Dispose();
if (_songOverlay is not null) await _songOverlay.DisposeAsync();
if (_lumiOverlay is not null) await _lumiOverlay.DisposeAsync();
await runtime.DisposeAsync();
desktop.Shutdown();
};
Program.InstanceCoordinator!.ActivationRequested += () => Dispatcher.UIThread.Post(ShowMainWindow);
if (!Program.LaunchInBackground) _mainWindow.Show();
_ = InitializeRuntimesAsync(runtime, _songOverlay);
_ = InitializeRuntimesAsync(runtime, _songOverlay, _lumiOverlay);
}
base.OnFrameworkInitializationCompleted();
}
private static async Task InitializeRuntimesAsync(CompanionRuntime runtime, SongOverlayRuntime songOverlay)
private static async Task InitializeRuntimesAsync(CompanionRuntime runtime, SongOverlayRuntime songOverlay, LumiOverlayRuntime lumiOverlay)
{
await runtime.InitializeAsync();
try { await songOverlay.InitializeAsync(); }
catch { /* The plugin publishes its own actionable status without taking down the shell. */ }
try { await lumiOverlay.InitializeAsync(); }
catch { /* Native overlay failures remain isolated and visible in the plugin status. */ }
}
private void ConfigureTray(IClassicDesktopStyleApplicationLifetime desktop, CompanionRuntime runtime, IReadOnlyList<ICompanionPluginContribution> plugins)
@ -116,6 +124,7 @@ public partial class App : Application
_mainWindow.AllowExit();
_trayIcon?.Dispose();
if (_songOverlay is not null) await _songOverlay.DisposeAsync();
if (_lumiOverlay is not null) await _lumiOverlay.DisposeAsync();
await runtime.DisposeAsync();
desktop.Shutdown();
};

File diff suppressed because it is too large Load Diff

View File

@ -6,11 +6,14 @@ namespace Lumi.Companion.App;
public sealed record CompanionSettings(
bool AutoStartWithWindows = false,
bool StartWithObs = true,
bool TranscriptionEnabled = true,
bool AdvancedMode = false,
string? PrimarySourceUuid = null,
string? PrimarySourceName = null,
DateTimeOffset? PathTestPassedAt = null,
string? PathTestFingerprint = null);
string? PathTestFingerprint = null,
int LastObsProcessId = 0,
bool StreamTestObsRestartRequired = false);
public sealed class CompanionSettingsStore
{

View File

@ -1,7 +1,7 @@
namespace Lumi.Companion.App;
public enum TrayHealth { Ready, Operating, Degraded, Failed }
public enum CompanionPage { Overview, Transcription, Test, SongOverlay, Connection, Logs, Settings }
public enum CompanionPage { Overview, StreamTesting, Transcription, Test, LumiOverlay, SongOverlay, Connection, Logs, Settings }
public enum TestStageState { Waiting, Running, Passed, Blocked, Failed }
public sealed record CompanionState(
@ -13,19 +13,33 @@ public sealed record CompanionState(
bool ObsRecording = false,
bool TestRunning = false,
bool BenchmarkRunning = false,
bool StreamTestRunning = false,
bool StreamTestRecoveryRequired = false,
bool StreamTestObsRestartRequired = false,
TrayHealth Health = TrayHealth.Degraded,
string Detail = "Pair Lumi Companion to get started.",
string? DeviceName = null,
string? Host = null,
DateTimeOffset? LastConnectedAt = null,
bool UpdateAvailable = false,
bool UpdateCheckRunning = false,
string? AvailableVersion = null,
string UpdateDetail = "Checking for updates…",
string BenchmarkDetail = "Start a dedicated test, speak naturally, then end it when you have enough material.",
string StreamTestDetail = "Start a private test only while OBS is not streaming or recording.",
string? StreamTestSessionId = null,
double StreamTestBitrateKbps = 0,
long StreamTestDroppedFrames = 0,
long StreamTestTotalFrames = 0,
double StreamTestCongestion = 0,
bool PathTestValid = false,
string PathTestDetail = "Run once after setup or a relevant configuration change.",
bool ObsBridgeRepairNeeded = false,
bool ObsBridgeUpdateAvailable = false,
bool ObsBridgePackageAvailable = false,
string? ObsBridgeInstalledVersion = null,
string? ObsBridgeLoadedVersion = null,
string? ObsBridgeBundledVersion = null,
string ObsBridgeDetail = "Checking the managed OBS integration…")
{
public string Summary => Health switch
@ -35,12 +49,15 @@ public sealed record CompanionState(
TrayHealth.Failed => "Needs attention",
_ when !Paired => "Setup required",
_ when !Connected => "Lumi offline",
_ when ObsBridgeUpdateAvailable => "OBS update required",
_ when !ObsBridgeInstalled => "OBS setup required",
_ => "Partially ready"
};
public bool RequiresQuitConfirmation => ObsStreaming || ObsRecording || BenchmarkRunning;
public string QuitWarning => BenchmarkRunning
public bool RequiresQuitConfirmation => ObsStreaming || ObsRecording || BenchmarkRunning || StreamTestRunning || StreamTestRecoveryRequired;
public string QuitWarning => StreamTestRunning || StreamTestRecoveryRequired
? "A private stream test or OBS recovery is active. Quitting will stop the test and restore the exact streaming service saved before it began."
: BenchmarkRunning
? "A transcription accuracy and latency test is running. Quitting will safely end and mark the test as aborted."
: ObsStreaming
? "OBS is streaming. Quitting Lumi Companion will stop transcription and closed captions, but it will not stop the OBS stream."

View File

@ -0,0 +1,134 @@
using Avalonia.Controls;
using Avalonia.Input;
using Avalonia.Threading;
namespace Lumi.Companion.App;
public sealed class HotkeyCaptureBinding
{
private readonly Button _button;
private readonly DispatcherTimer _timeout;
private readonly HashSet<Key> _pressed = [];
private string _hotkey = "";
private string _previous = "";
private KeyModifiers _modifiers;
private Key? _primary;
private bool _capturing;
public HotkeyCaptureBinding(Button button)
{
_button = button;
_button.HorizontalContentAlignment = Avalonia.Layout.HorizontalAlignment.Left;
_button.Focusable = true;
_timeout = new DispatcherTimer { Interval = TimeSpan.FromSeconds(7) };
_timeout.Tick += (_, _) => CancelCapture();
_button.Click += (_, _) => BeginCapture();
_button.KeyDown += OnKeyDown;
_button.KeyUp += OnKeyUp;
}
public string Hotkey
{
get => _hotkey;
set
{
_hotkey = string.IsNullOrWhiteSpace(value) ? "Unassigned" : value.Trim();
if (!_capturing) UpdateContent();
}
}
private void OnKeyDown(object? sender, KeyEventArgs e)
{
if (!_capturing)
{
return;
}
e.Handled = true;
if (e.Key == Key.Escape)
{
CancelCapture();
return;
}
_pressed.Add(e.Key);
_modifiers |= e.KeyModifiers;
if (!IsModifier(e.Key)) _primary = e.Key;
_button.Content = _primary is null ? "Hold modifiers, then press a key…" : $"{Format(_modifiers, _primary.Value)} · release to save";
}
private void OnKeyUp(object? sender, KeyEventArgs e)
{
if (!_capturing)
{
return;
}
e.Handled = true;
_pressed.Remove(e.Key);
if (_primary is not null && !_pressed.Any(key => !IsModifier(key)))
{
_hotkey = Format(_modifiers, _primary.Value);
FinishCapture();
}
}
private void BeginCapture()
{
if (_capturing) return;
_capturing = true;
_previous = _hotkey;
_pressed.Clear();
_modifiers = KeyModifiers.None;
_primary = null;
_button.Content = "Press a shortcut…";
_button.Focus();
_timeout.Start();
}
private void CancelCapture()
{
if (!_capturing) return;
_hotkey = _previous;
FinishCapture();
}
private void FinishCapture()
{
_capturing = false;
_timeout.Stop();
_pressed.Clear();
UpdateContent();
}
private void UpdateContent() => _button.Content = $"{_hotkey} (Set hotkey)";
private static bool IsModifier(Key key) => key is
Key.LeftCtrl or Key.RightCtrl or Key.LeftAlt or Key.RightAlt or
Key.LeftShift or Key.RightShift or Key.LWin or Key.RWin;
internal static string Format(KeyModifiers modifiers, Key key)
{
var parts = new List<string>(5);
if (modifiers.HasFlag(KeyModifiers.Control)) parts.Add("Ctrl");
if (modifiers.HasFlag(KeyModifiers.Alt)) parts.Add("Alt");
if (modifiers.HasFlag(KeyModifiers.Shift)) parts.Add("Shift");
if (modifiers.HasFlag(KeyModifiers.Meta)) parts.Add("Win");
parts.Add(KeyName(key));
return string.Join("+", parts);
}
private static string KeyName(Key key) => key switch
{
Key.PageUp => "PageUp",
Key.PageDown => "PageDown",
Key.Left => "Left",
Key.Right => "Right",
Key.Up => "Up",
Key.Down => "Down",
Key.Return => "Enter",
Key.Space => "Space",
Key.Back => "Backspace",
_ => key.ToString()
};
}

View File

@ -6,8 +6,8 @@
<ImplicitUsings>enable</ImplicitUsings>
<ApplicationManifest>app.manifest</ApplicationManifest>
<ApplicationIcon>Assets\Lumi.Companion.ico</ApplicationIcon>
<Version>0.1.0</Version>
<AssemblyVersion>0.1.0.0</AssemblyVersion>
<Version>0.2.8</Version>
<AssemblyVersion>0.2.8.0</AssemblyVersion>
</PropertyGroup>
<ItemGroup>
<AvaloniaResource Include="Assets\Lumi.Companion.ico" />
@ -16,6 +16,7 @@
<ProjectReference Include="../Lumi.Companion.Abstractions/Lumi.Companion.Abstractions.csproj" />
<ProjectReference Include="../../plugins/Lumi.Companion.Transcription/Lumi.Companion.Transcription.csproj" />
<ProjectReference Include="../../plugins/Lumi.Companion.SongOverlay/Lumi.Companion.SongOverlay.csproj" />
<ProjectReference Include="../../plugins/Lumi.Companion.Overlay/Lumi.Companion.Overlay.csproj" />
<PackageReference Include="Avalonia" Version="12.1.0" />
<PackageReference Include="Avalonia.Desktop" Version="12.1.0" />
<PackageReference Include="Avalonia.Fonts.Inter" Version="12.1.0" />

View File

@ -29,7 +29,10 @@
</Border>
<Grid Grid.Row="2" RowDefinitions="Auto,*,Auto">
<Button x:Name="OverviewNav" Classes="nav selected" Content="Overview" Tag="Overview" />
<StackPanel Spacing="4">
<Button x:Name="OverviewNav" Classes="nav selected" Content="Overview" Tag="Overview" />
<Button x:Name="StreamTestingNav" Classes="nav" Content="Stream testing" Tag="StreamTesting" />
</StackPanel>
<ScrollViewer Grid.Row="1" Margin="0,14,0,14" VerticalScrollBarVisibility="Auto">
<StackPanel Spacing="6">
@ -79,6 +82,16 @@
</Grid>
</Border>
<Border x:Name="OverviewBridgeUpdatePanel" Background="#FFF2D6" BorderBrush="#D58A1F" BorderThickness="1" CornerRadius="12" Padding="18" IsVisible="False">
<Grid ColumnDefinitions="*,Auto" ColumnSpacing="20">
<StackPanel Spacing="4">
<TextBlock Text="Update OBS Bridge" FontWeight="Bold" Foreground="#764600" />
<TextBlock x:Name="OverviewBridgeUpdateDetail" Text="The OBS integration loaded by OBS is older than the version bundled with Companion." TextWrapping="Wrap" Foreground="#764600" />
</StackPanel>
<Button Grid.Column="1" x:Name="OverviewUpdateBridgeButton" Classes="primary" Content="Update OBS Bridge" VerticalAlignment="Center" />
</Grid>
</Border>
<StackPanel Spacing="12">
<TextBlock Text="Setup progress" Classes="sectionTitle" />
<Grid ColumnDefinitions="*,*,*" ColumnSpacing="12">
@ -112,6 +125,47 @@
</StackPanel>
</ScrollViewer>
<ScrollViewer x:Name="StreamTestingPage" IsVisible="False">
<StackPanel Margin="44,38,52,48" MaxWidth="760" HorizontalAlignment="Left" Spacing="24">
<StackPanel Spacing="6">
<TextBlock Text="PRIVATE OUTPUT" Classes="eyebrow" />
<TextBlock Text="Stream testing" Classes="pageTitle" />
<TextBlock Text="Send the real OBS stream to Lumi's private test receiver, inspect it in the Admin page, then restore the exact normal streaming service automatically." Classes="muted" FontSize="15" />
</StackPanel>
<Border Background="#FFF2D6" BorderBrush="#D58A1F" BorderThickness="1" CornerRadius="12" Padding="18">
<StackPanel Spacing="5">
<TextBlock Text="Never start this while publicly live" FontWeight="Bold" Foreground="#764600" />
<TextBlock Text="Companion refuses to redirect active streaming or recording. While the test is active, the status below remains unmistakable and the prior OBS service is kept in an encrypted current-user recovery file." TextWrapping="Wrap" Foreground="#764600" />
</StackPanel>
</Border>
<Border Classes="card">
<StackPanel Spacing="16">
<StackPanel Spacing="5">
<TextBlock x:Name="StreamTestStatusTitle" Text="Ready to check" Classes="sectionTitle" />
<TextBlock x:Name="StreamTestStatusDetail" Text="Start only while OBS is idle." Classes="muted" TextWrapping="Wrap" />
</StackPanel>
<Grid ColumnDefinitions="*,*,*" ColumnSpacing="12">
<Border Classes="soft"><StackPanel Spacing="4"><TextBlock Text="OBS bitrate" Classes="muted" FontSize="11" /><TextBlock x:Name="StreamTestBitrateText" Text="—" FontWeight="SemiBold" /></StackPanel></Border>
<Border Grid.Column="1" Classes="soft"><StackPanel Spacing="4"><TextBlock Text="Dropped frames" Classes="muted" FontSize="11" /><TextBlock x:Name="StreamTestDroppedText" Text="—" FontWeight="SemiBold" /></StackPanel></Border>
<Border Grid.Column="2" Classes="soft"><StackPanel Spacing="4"><TextBlock Text="Congestion" Classes="muted" FontSize="11" /><TextBlock x:Name="StreamTestCongestionText" Text="—" FontWeight="SemiBold" /></StackPanel></Border>
</Grid>
<StackPanel Orientation="Horizontal" Spacing="10">
<Button x:Name="StartStreamTestButton" Classes="primary" Content="Start private stream test" />
<Button x:Name="StopStreamTestButton" Classes="secondary" Content="End test and restore OBS" IsEnabled="False" />
<Button x:Name="OpenStreamTestWebButton" Classes="secondary" Content="Open stream viewer" IsVisible="False" />
<Button x:Name="RepairStreamTestButton" Classes="secondary" Content="Repair restoration" IsVisible="False" />
</StackPanel>
</StackPanel>
</Border>
<Border Classes="soft">
<StackPanel Spacing="5">
<TextBlock Text="Where to watch" FontWeight="SemiBold" />
<TextBlock Text="Open Admin → Stream testing in Lumi WebUI for the private player, quality control, receiver facts, warnings, and recovery timeline." Classes="muted" TextWrapping="Wrap" />
</StackPanel>
</Border>
</StackPanel>
</ScrollViewer>
<ScrollViewer x:Name="TranscriptionPage" IsVisible="False">
<StackPanel Margin="44,38,52,48" MaxWidth="760" HorizontalAlignment="Left" Spacing="25">
<StackPanel Spacing="6">
@ -119,6 +173,15 @@
<TextBlock Text="Transcription" Classes="pageTitle" />
<TextBlock Text="Choose the OBS microphone Lumi should caption. Source identity remains stable if you rename it." Classes="muted" FontSize="15" />
</StackPanel>
<Border Classes="card">
<Grid ColumnDefinitions="*,Auto" ColumnSpacing="24">
<StackPanel Spacing="4">
<TextBlock Text="Generate and include captions" FontWeight="SemiBold" />
<TextBlock x:Name="TranscriptionEnabledDetail" Text="Speech recognition and caption inclusion are enabled." Classes="muted" TextWrapping="Wrap" />
</StackPanel>
<ToggleSwitch Grid.Column="1" x:Name="TranscriptionEnabledToggle" OffContent="Off" OnContent="On" VerticalAlignment="Center" />
</Grid>
</Border>
<StackPanel Spacing="8">
<TextBlock Text="Primary microphone" FontWeight="SemiBold" />
<ComboBox x:Name="SourcePicker" IsEnabled="False" PlaceholderText="Open OBS after installing the managed integration" MinWidth="420" HorizontalAlignment="Left" />
@ -189,7 +252,7 @@
<StackPanel Spacing="6">
<TextBlock Text="MEDIA" Classes="eyebrow" />
<TextBlock Text="Song Overlay" Classes="pageTitle" />
<TextBlock Text="Spotify is the first media provider. The provider contract remains generic so later services can use the same event and overlay pipeline." Classes="muted" FontSize="15" />
<TextBlock Text="Choose which Windows media sessions Lumi may discover. Song delivery continues to use the Companion's shared secure connection." Classes="muted" FontSize="15" TextWrapping="Wrap" />
</StackPanel>
<Border Classes="soft">
@ -214,16 +277,16 @@
</StackPanel>
<StackPanel Spacing="12">
<TextBlock Text="Playback provider" Classes="sectionTitle" />
<Grid ColumnDefinitions="150,*" RowDefinitions="Auto,Auto,Auto" RowSpacing="10" ColumnSpacing="12">
<TextBlock Text="Provider" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Column="1" x:Name="SongOverlayProviderPicker" SelectedIndex="0" IsEnabled="False">
<ComboBoxItem Content="Spotify" Tag="spotify" />
</ComboBox>
<TextBlock Grid.Row="1" Text="Recovery heartbeat" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="1" Grid.Column="1" x:Name="SongOverlayHeartbeatBox" Minimum="15" Maximum="300" Increment="5" FormatString="0 seconds" HorizontalAlignment="Left" Width="160" />
<TextBlock Grid.Row="2" Text="Seek sensitivity" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="2" Grid.Column="1" x:Name="SongOverlaySeekBox" Minimum="500" Maximum="10000" Increment="250" FormatString="0 ms" HorizontalAlignment="Left" Width="160" />
<TextBlock Text="Playback sources" Classes="sectionTitle" />
<TextBlock Text="Only enabled sources are observed. Browser sessions without an exposed service name are treated as YouTube." Classes="muted" FontSize="12" TextWrapping="Wrap" />
<Border Classes="soft">
<StackPanel x:Name="SongOverlaySourcesPanel" Spacing="9" />
</Border>
<Grid ColumnDefinitions="150,*" RowDefinitions="Auto,Auto" RowSpacing="10" ColumnSpacing="12">
<TextBlock Text="Recovery heartbeat" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Column="1" x:Name="SongOverlayHeartbeatBox" Minimum="15" Maximum="300" Increment="5" FormatString="0 seconds" HorizontalAlignment="Left" Width="160" />
<TextBlock Grid.Row="1" Text="Seek sensitivity" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="1" Grid.Column="1" x:Name="SongOverlaySeekBox" Minimum="500" Maximum="10000" Increment="250" FormatString="0 ms" HorizontalAlignment="Left" Width="160" />
</Grid>
<CheckBox x:Name="SongOverlayCoverToggle" Content="Send cover art only with track metadata" />
<CheckBox x:Name="SongOverlaySearchLinkToggle" Content="Use a Spotify search link when an exact song link is unavailable" />
@ -253,6 +316,189 @@
</StackPanel>
</ScrollViewer>
<ScrollViewer x:Name="LumiOverlayPage" IsVisible="False">
<StackPanel Margin="44,38,52,48" MaxWidth="820" HorizontalAlignment="Left" Spacing="24">
<StackPanel Spacing="6">
<TextBlock Text="NATIVE MONITOR OVERLAY" Classes="eyebrow" />
<TextBlock Text="Lumi Overlay" Classes="pageTitle" />
<TextBlock Text="Show Lumi's normalized chat and stream events directly on one Windows monitor. This is separate from browser-source overlays and never receives provider credentials." Classes="muted" FontSize="15" TextWrapping="Wrap" />
<TextBlock Text="Changes on this page stay as an unsaved draft until you select Save Lumi Overlay. Live feed refreshes will not replace the draft." Classes="muted" FontSize="12" TextWrapping="Wrap" />
</StackPanel>
<Grid ColumnDefinitions="*,*" ColumnSpacing="12">
<Border Classes="soft">
<StackPanel Spacing="4">
<TextBlock Text="Feed" Classes="muted" FontSize="11" />
<TextBlock x:Name="LumiOverlayFeedStatus" Text="Starting" FontWeight="SemiBold" />
<TextBlock x:Name="LumiOverlayFeedDetail" Text="Loading the native overlay…" Classes="muted" TextWrapping="Wrap" />
<TextBlock x:Name="LumiOverlayProviderStatus" Text="Provider status will appear after Lumi connects." Classes="muted" FontSize="11" TextWrapping="Wrap" />
</StackPanel>
</Border>
<Border Grid.Column="1" Classes="soft">
<StackPanel Spacing="4">
<TextBlock Text="Capture exclusion" Classes="muted" FontSize="11" />
<TextBlock x:Name="LumiOverlayCaptureStatus" Text="Pending" FontWeight="SemiBold" />
<TextBlock x:Name="LumiOverlayCaptureDetail" Text="Waiting for the overlay window." Classes="muted" TextWrapping="Wrap" />
</StackPanel>
</Border>
</Grid>
<TextBlock x:Name="LumiOverlayMonitorWarning" Foreground="{DynamicResource LumiWarningBrush}" TextWrapping="Wrap" IsVisible="False" />
<Border Classes="card">
<StackPanel Spacing="14">
<TextBlock Text="Display &amp; visibility" Classes="sectionTitle" />
<Grid ColumnDefinitions="180,*" RowDefinitions="Auto,Auto,Auto,Auto" RowSpacing="10" ColumnSpacing="12">
<TextBlock Text="Monitor" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Column="1" x:Name="LumiOverlayMonitorPicker" />
<TextBlock Grid.Row="1" Text="Visibility" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Row="1" Grid.Column="1" x:Name="LumiOverlayVisibilityPicker" />
<TextBlock Grid.Row="2" Text="Layout" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Row="2" Grid.Column="1" x:Name="LumiOverlayLayoutPicker" />
<TextBlock Grid.Row="3" x:Name="LumiOverlayChatShareLabel" Text="Chat share in divided layout" Classes="muted" VerticalAlignment="Center" />
<StackPanel Grid.Row="3" Grid.Column="1" x:Name="LumiOverlayChatShareEditor" Orientation="Horizontal" Spacing="7">
<NumericUpDown x:Name="LumiOverlayChatPercentageBox" Minimum="10" Maximum="90" Increment="5" FormatString="0" HorizontalAlignment="Left" Width="112" />
<TextBlock Text="%" Classes="muted" VerticalAlignment="Center" />
</StackPanel>
</Grid>
<CheckBox x:Name="LumiOverlayEnabledToggle" Content="Run Lumi Overlay automatically with Companion" />
<TextBlock x:Name="LumiOverlayVisibilityState" Text="Current display: checking…" Classes="muted" FontSize="12" TextWrapping="Wrap" />
<StackPanel Orientation="Horizontal" Spacing="10">
<Button x:Name="LumiOverlayPreviewButton" Classes="secondary" Content="Start sample preview" />
<Button x:Name="LumiOverlayVisibilityButton" Classes="secondary" Content="Show temporarily" />
</StackPanel>
</StackPanel>
</Border>
<Grid ColumnDefinitions="*,*" ColumnSpacing="16">
<StackPanel Spacing="10">
<TextBlock Text="Sources" Classes="sectionTitle" />
<TextBlock Text="Only checked sources are sent over this device feed. Saved unavailable sources stay listed." Classes="muted" FontSize="12" TextWrapping="Wrap" />
<Border Classes="soft"><StackPanel x:Name="LumiOverlaySourcesPanel" Spacing="8"><TextBlock Text="Waiting for Lumi…" Classes="muted" /></StackPanel></Border>
</StackPanel>
<StackPanel Grid.Column="1" Spacing="10">
<TextBlock Text="Event types" Classes="sectionTitle" />
<TextBlock Text="Chat is controlled by Sources. Choose which non-chat events should appear." Classes="muted" FontSize="12" TextWrapping="Wrap" />
<Border Classes="soft"><StackPanel x:Name="LumiOverlayEventsPanel" Spacing="8"><TextBlock Text="Waiting for Lumi…" Classes="muted" /></StackPanel></Border>
</StackPanel>
</Grid>
<Border Classes="card">
<Grid ColumnDefinitions="*,Auto" ColumnSpacing="20">
<StackPanel Spacing="5">
<TextBlock Text="Interactive layout editor" Classes="sectionTitle" />
<TextBlock Text="Edit the real monitor overlay in place. Drag a highlighted container to move it, or drag its lower-right corner to resize it. The preview uses the same renderer, typography, media and animations as the live overlay." Classes="muted" TextWrapping="Wrap" />
<TextBlock x:Name="LumiOverlayEditStatus" Text="Edit mode is off. Changes remain a draft until saved." Classes="muted" FontSize="12" TextWrapping="Wrap" />
</StackPanel>
<Button Grid.Column="1" x:Name="LumiOverlayEditButton" Classes="primary" Content="Start edit mode" VerticalAlignment="Center" />
</Grid>
</Border>
<Expander Header="Advanced appearance &amp; lifecycle">
<Border Classes="card" Margin="0,10,0,0">
<StackPanel Spacing="14">
<TextBlock Text="Container styling &amp; lifecycle" Classes="sectionTitle" />
<ComboBox x:Name="LumiOverlayContainerPicker" HorizontalAlignment="Left" Width="240">
<ComboBoxItem x:Name="LumiOverlayUnifiedContainerOption" Content="Unified container" Tag="unified" />
<ComboBoxItem x:Name="LumiOverlayChatContainerOption" Content="Chat container" Tag="chat" />
<ComboBoxItem x:Name="LumiOverlayEventContainerOption" Content="Event container" Tag="events" />
</ComboBox>
<Grid ColumnDefinitions="170,180,170,180" RowDefinitions="Auto,Auto,Auto,Auto,Auto,Auto" RowSpacing="10" ColumnSpacing="10">
<TextBlock Text="Anchor" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Column="1" x:Name="LumiOverlayAnchorPicker" />
<TextBlock Grid.Column="2" Text="Newest items" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Column="3" x:Name="LumiOverlayDirectionPicker" />
<TextBlock Grid.Row="1" Text="Width (physical px)" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="1" Grid.Column="1" x:Name="LumiOverlayWidthBox" Minimum="160" Maximum="16000" />
<TextBlock Grid.Row="1" Grid.Column="2" Text="Height (physical px)" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="1" Grid.Column="3" x:Name="LumiOverlayHeightBox" Minimum="100" Maximum="16000" />
<TextBlock Grid.Row="2" Text="Card RGBA" Classes="muted" VerticalAlignment="Center" />
<TextBox Grid.Row="2" Grid.Column="1" x:Name="LumiOverlayCardColorBox" />
<TextBlock Grid.Row="2" Grid.Column="2" Text="Text RGB" Classes="muted" VerticalAlignment="Center" />
<TextBox Grid.Row="2" Grid.Column="3" x:Name="LumiOverlayTextColorBox" />
<TextBlock Grid.Row="3" Text="Font" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Row="3" Grid.Column="1" x:Name="LumiOverlayFontBox" />
<TextBlock Grid.Row="3" Grid.Column="2" Text="Font size" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="3" Grid.Column="3" x:Name="LumiOverlayFontSizeBox" Minimum="8" Maximum="120" />
<TextBlock Grid.Row="4" Text="Entry animation" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Row="4" Grid.Column="1" x:Name="LumiOverlayEntryPicker" />
<TextBlock Grid.Row="4" Grid.Column="2" Text="Exit animation" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Row="4" Grid.Column="3" x:Name="LumiOverlayExitPicker" />
<TextBlock Grid.Row="5" Text="Lifetime (ms)" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="5" Grid.Column="1" x:Name="LumiOverlayLifetimeBox" Minimum="1000" Maximum="300000" Increment="1000" />
<TextBlock Grid.Row="5" Grid.Column="2" Text="Queue limit" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="5" Grid.Column="3" x:Name="LumiOverlayQueueBox" Minimum="1" Maximum="100" />
</Grid>
<StackPanel Orientation="Horizontal" Spacing="14">
<CheckBox x:Name="LumiOverlayAvatarToggle" Content="Avatar" />
<CheckBox x:Name="LumiOverlayUsernameToggle" Content="Username" />
<CheckBox x:Name="LumiOverlayPlatformToggle" Content="Platform identity" />
</StackPanel>
<Expander Header="Position, shadow &amp; animation details">
<StackPanel Spacing="12" Margin="0,12,0,0">
<Grid ColumnDefinitions="130,120,130,120" RowDefinitions="Auto,Auto,Auto" RowSpacing="8" ColumnSpacing="10">
<TextBlock Text="Top padding" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Column="1" x:Name="LumiOverlayPaddingTopBox" Minimum="0" Maximum="4000" />
<TextBlock Grid.Column="2" Text="Right padding" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Column="3" x:Name="LumiOverlayPaddingRightBox" Minimum="0" Maximum="4000" />
<TextBlock Grid.Row="1" Text="Bottom padding" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="1" Grid.Column="1" x:Name="LumiOverlayPaddingBottomBox" Minimum="0" Maximum="4000" />
<TextBlock Grid.Row="1" Grid.Column="2" Text="Left padding" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="1" Grid.Column="3" x:Name="LumiOverlayPaddingLeftBox" Minimum="0" Maximum="4000" />
<TextBlock Grid.Row="2" Text="Horizontal align" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Row="2" Grid.Column="1" x:Name="LumiOverlayHorizontalAlignPicker" />
<TextBlock Grid.Row="2" Grid.Column="2" Text="Vertical align" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Row="2" Grid.Column="3" x:Name="LumiOverlayVerticalAlignPicker" />
</Grid>
<Grid ColumnDefinitions="130,120,130,120" RowDefinitions="Auto,Auto" RowSpacing="8" ColumnSpacing="10">
<TextBlock Text="Shadow RGBA" Classes="muted" VerticalAlignment="Center" />
<TextBox Grid.Column="1" x:Name="LumiOverlayShadowColorBox" />
<TextBlock Grid.Column="2" Text="Shadow blur" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Column="3" x:Name="LumiOverlayShadowBlurBox" Minimum="0" Maximum="40" />
<TextBlock Grid.Row="1" Text="Shadow X / Y" Classes="muted" VerticalAlignment="Center" />
<StackPanel Grid.Row="1" Grid.Column="1" Orientation="Horizontal" Spacing="6">
<NumericUpDown x:Name="LumiOverlayShadowXBox" Minimum="-40" Maximum="40" Width="56" />
<NumericUpDown x:Name="LumiOverlayShadowYBox" Minimum="-40" Maximum="40" Width="56" />
</StackPanel>
<TextBlock Grid.Row="1" Grid.Column="2" Text="Easing" Classes="muted" VerticalAlignment="Center" />
<ComboBox Grid.Row="1" Grid.Column="3" x:Name="LumiOverlayEasingPicker" />
</Grid>
<Grid ColumnDefinitions="130,120,130,120" RowDefinitions="Auto,Auto" RowSpacing="8" ColumnSpacing="10">
<TextBlock Text="Entry duration" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Column="1" x:Name="LumiOverlayEntryDurationBox" Minimum="0" Maximum="5000" Increment="20" FormatString="0 ms" />
<TextBlock Grid.Column="2" Text="Exit duration" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Column="3" x:Name="LumiOverlayExitDurationBox" Minimum="0" Maximum="5000" Increment="20" FormatString="0 ms" />
<TextBlock Grid.Row="1" Text="Slide distance" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="1" Grid.Column="1" x:Name="LumiOverlaySlideDistanceBox" Minimum="0" Maximum="1000" />
<TextBlock Grid.Row="1" Grid.Column="2" Text="Scale amount" Classes="muted" VerticalAlignment="Center" />
<NumericUpDown Grid.Row="1" Grid.Column="3" x:Name="LumiOverlayScaleAmountBox" Minimum="0.1" Maximum="2" Increment="0.05" FormatString="0.00" />
</Grid>
</StackPanel>
</Expander>
<TextBlock Text="Disabled padding edges are always stored as zero. Width and height are physical pixels and clamp to the selected monitor." Classes="muted" FontSize="12" TextWrapping="Wrap" />
</StackPanel>
</Border>
</Expander>
<Border Classes="soft">
<StackPanel Spacing="12">
<TextBlock Text="Global hotkeys" Classes="sectionTitle" />
<Grid ColumnDefinitions="180,*" RowDefinitions="Auto,Auto" RowSpacing="10">
<TextBlock Text="Visibility override" Classes="muted" />
<Button Grid.Column="1" x:Name="LumiOverlayVisibilityHotkeyBox" Classes="secondary" />
<TextBlock Grid.Row="1" Text="Preview override" Classes="muted" />
<Button Grid.Row="1" Grid.Column="1" x:Name="LumiOverlayPreviewHotkeyBox" Classes="secondary" />
</Grid>
<TextBlock x:Name="LumiOverlayHotkeyStatus" Text="Hotkeys are not registered yet." Classes="muted" FontSize="12" TextWrapping="Wrap" />
</StackPanel>
</Border>
<StackPanel Orientation="Horizontal" Spacing="10">
<Button x:Name="SaveLumiOverlayButton" Classes="primary" Content="Save Lumi Overlay" />
</StackPanel>
<TextBlock x:Name="LumiOverlayFeedback" Classes="muted" TextWrapping="Wrap" />
</StackPanel>
</ScrollViewer>
<ScrollViewer x:Name="ConnectionPage" IsVisible="False">
<StackPanel Margin="44,38,52,48" MaxWidth="760" HorizontalAlignment="Left" Spacing="24">
<StackPanel Spacing="6">
@ -270,6 +516,15 @@
<Button x:Name="PairButton" Classes="secondary" Content="Choose pairing package" />
<Button x:Name="OpenWebButton" Classes="secondary" Content="Open Lumi WebUI" />
</StackPanel>
<Border x:Name="ConnectionBridgeUpdatePanel" Background="#FFF2D6" BorderBrush="#D58A1F" BorderThickness="1" CornerRadius="12" Padding="18" IsVisible="False">
<Grid ColumnDefinitions="*,Auto" ColumnSpacing="20">
<StackPanel Spacing="4">
<TextBlock Text="OBS Bridge update required" FontWeight="Bold" Foreground="#764600" />
<TextBlock x:Name="ConnectionBridgeUpdateDetail" Text="Close OBS before updating the managed integration." TextWrapping="Wrap" Foreground="#764600" />
</StackPanel>
<Button Grid.Column="1" x:Name="ConnectionUpdateBridgeButton" Classes="primary" Content="Update OBS Bridge" VerticalAlignment="Center" />
</Grid>
</Border>
<Border Classes="soft">
<StackPanel Spacing="10">
<TextBlock Text="Managed OBS integration" FontWeight="SemiBold" />

View File

@ -6,6 +6,7 @@ using Avalonia.Platform.Storage;
using Avalonia.Threading;
using Lumi.Companion.Abstractions;
using Lumi.Companion.SongOverlay;
using Lumi.Companion.Overlay;
namespace Lumi.Companion.App;
@ -14,30 +15,47 @@ public partial class MainWindow : Window
private readonly CompanionRuntime _runtime;
private readonly CompanionSettingsStore _settings;
private readonly SongOverlayRuntime _songOverlay;
private readonly LumiOverlayRuntime _lumiOverlay;
private readonly IReadOnlyList<ICompanionPluginContribution> _plugins;
private readonly Dictionary<CompanionPage, Button> _navigationButtons = [];
private readonly Dictionary<CompanionPage, PluginNavigationGroup> _pluginNavigationGroups = [];
private bool _allowExit;
private bool _renderingSources;
private bool _renderingSongOverlay;
private bool _renderingLumiOverlay;
private bool _renderingSettings;
private readonly Dictionary<string, CheckBox> _overlaySourceChecks = [];
private readonly Dictionary<string, CheckBox> _overlayEventChecks = [];
private readonly Dictionary<string, CheckBox> _songOverlaySourceChecks = [];
private bool _songOverlaySourcesDirty;
private string _editingOverlayContainer = "unified";
private LumiOverlaySettings? _lumiOverlayDraft;
private readonly HotkeyCaptureBinding _visibilityHotkeyCapture;
private readonly HotkeyCaptureBinding _previewHotkeyCapture;
public MainWindow() : this(CreateDefaultServices()) { }
private MainWindow((CompanionRuntime Runtime, CompanionSettingsStore Settings, SongOverlayRuntime SongOverlay, IReadOnlyList<ICompanionPluginContribution> Plugins) services)
: this(services.Runtime, services.Settings, services.SongOverlay, services.Plugins) { }
private MainWindow((CompanionRuntime Runtime, CompanionSettingsStore Settings, SongOverlayRuntime SongOverlay, LumiOverlayRuntime LumiOverlay, IReadOnlyList<ICompanionPluginContribution> Plugins) services)
: this(services.Runtime, services.Settings, services.SongOverlay, services.LumiOverlay, services.Plugins) { }
public MainWindow(CompanionRuntime runtime, CompanionSettingsStore settings, SongOverlayRuntime songOverlay, IReadOnlyList<ICompanionPluginContribution> plugins)
public MainWindow(CompanionRuntime runtime, CompanionSettingsStore settings, SongOverlayRuntime songOverlay, LumiOverlayRuntime lumiOverlay, IReadOnlyList<ICompanionPluginContribution> plugins)
{
_runtime = runtime;
_settings = settings;
_songOverlay = songOverlay;
_lumiOverlay = lumiOverlay;
_plugins = plugins;
InitializeComponent();
_visibilityHotkeyCapture = new HotkeyCaptureBinding(LumiOverlayVisibilityHotkeyBox);
_previewHotkeyCapture = new HotkeyCaptureBinding(LumiOverlayPreviewHotkeyBox);
BuildPluginNavigation();
WireActions();
RenderState(runtime.State);
RenderTestStages(runtime.TestStages);
RenderBenchmark(runtime.Benchmark);
RenderSongOverlay();
PrepareLumiOverlayControls();
RenderLumiOverlay();
Closing += OnClosing;
runtime.StateChanged += state => Dispatcher.UIThread.Post(() => { RenderState(state); RenderSongOverlay(); });
runtime.TestStagesChanged += stages => Dispatcher.UIThread.Post(() => RenderTestStages(stages));
@ -49,8 +67,9 @@ public partial class MainWindow : Window
if (simulated) SimulatedCaption.Text = text;
});
runtime.LogAdded += line => Dispatcher.UIThread.Post(() => AddLog(line));
settings.Changed += value => Dispatcher.UIThread.Post(() => RenderSettings(value));
settings.Changed += value => Dispatcher.UIThread.Post(() => { RenderSettings(value); RenderState(runtime.State); });
songOverlay.Changed += () => Dispatcher.UIThread.Post(RenderSongOverlay);
lumiOverlay.Changed += () => Dispatcher.UIThread.Post(() => RenderLumiOverlay());
}
private void BuildPluginNavigation()
@ -58,27 +77,73 @@ public partial class MainWindow : Window
PluginNavigation.Children.Clear();
foreach (var plugin in _plugins.OrderBy(item => item.Descriptor.Order).ThenBy(item => item.Descriptor.Name, StringComparer.OrdinalIgnoreCase))
{
var children = new StackPanel { Spacing = 2, Margin = new Thickness(8, 3, 0, 6) };
var pages = new List<(CompanionPluginPage Page, CompanionPage Parsed)>();
foreach (var page in plugin.Pages.OrderBy(item => item.Order))
if (Enum.TryParse<CompanionPage>(page.Key, out var parsed))
pages.Add((page, parsed));
if (pages.Count == 0) continue;
if (!CompanionPluginNavigation.UsesNestedNavigation(pages.Select(item => item.Page).ToList()))
{
var only = pages[0];
var button = new Button
{
Content = plugin.Descriptor.Name,
Tag = only.Page.Key,
HorizontalContentAlignment = Avalonia.Layout.HorizontalAlignment.Left
};
button.Classes.Add("nav");
button.Click += OnNavigate;
PluginNavigation.Children.Add(button);
_navigationButtons[only.Parsed] = button;
continue;
}
var children = new StackPanel
{
Spacing = 2,
Margin = new Thickness(8, 1, 0, 6),
IsVisible = false
};
var arrow = new TextBlock
{
Text = "",
FontSize = 17,
Foreground = new SolidColorBrush(Color.Parse("#5A6872")),
VerticalAlignment = Avalonia.Layout.VerticalAlignment.Center
};
var header = new Grid { ColumnDefinitions = ColumnDefinitions.Parse("*,Auto") };
header.Children.Add(new TextBlock
{
Text = plugin.Descriptor.Name,
VerticalAlignment = Avalonia.Layout.VerticalAlignment.Center
});
Grid.SetColumn(arrow, 1);
header.Children.Add(arrow);
var root = new Button
{
Content = header,
HorizontalContentAlignment = Avalonia.Layout.HorizontalAlignment.Stretch
};
root.Classes.Add("nav");
root.Classes.Add("navRoot");
root.Click += (_, _) => SetPluginNavigationExpanded(children, arrow, !children.IsVisible);
foreach (var (page, parsed) in pages)
{
if (!Enum.TryParse<CompanionPage>(page.Key, out var parsed)) continue;
var button = new Button { Content = page.Label, Tag = page.Key, HorizontalContentAlignment = Avalonia.Layout.HorizontalAlignment.Left };
button.Classes.Add("nav");
button.Click += OnNavigate;
children.Children.Add(button);
_navigationButtons[parsed] = button;
_pluginNavigationGroups[parsed] = new PluginNavigationGroup(children, arrow);
}
var expander = new Expander
{
Header = new TextBlock { Text = plugin.Descriptor.Name, FontWeight = FontWeight.SemiBold },
IsExpanded = true,
Content = children,
HorizontalAlignment = Avalonia.Layout.HorizontalAlignment.Stretch
};
expander.Classes.Add("pluginRoot");
PluginNavigation.Children.Add(expander);
var group = new StackPanel { Spacing = 1 };
group.Children.Add(root);
group.Children.Add(children);
PluginNavigation.Children.Add(group);
}
_navigationButtons[CompanionPage.Overview] = OverviewNav;
_navigationButtons[CompanionPage.StreamTesting] = StreamTestingNav;
_navigationButtons[CompanionPage.Connection] = ConnectionNav;
_navigationButtons[CompanionPage.Logs] = LogsNav;
_navigationButtons[CompanionPage.Settings] = SettingsNav;
@ -87,6 +152,7 @@ public partial class MainWindow : Window
private void WireActions()
{
OverviewNav.Click += OnNavigate;
StreamTestingNav.Click += OnNavigate;
foreach (var button in TechnicalNavigation.Children.OfType<Button>()) button.Click += OnNavigate;
NextActionButton.Click += OnNextAction;
PairButton.Click += async (_, _) => await ChoosePairingPackageAsync();
@ -94,16 +160,46 @@ public partial class MainWindow : Window
OpenWebButton.Click += (_, _) => _runtime.OpenLumiWebUi();
OpenLogsButton.Click += (_, _) => _runtime.OpenLogsDirectory();
RunTestButton.Click += async (_, _) => await RunUiActionAsync(() => _runtime.RunTestAsync(), RunTestButton);
StartStreamTestButton.Click += async (_, _) => await RunUiActionAsync(() => _runtime.StartStreamTestAsync(), StartStreamTestButton);
StopStreamTestButton.Click += async (_, _) => await RunUiActionAsync(() => _runtime.StopStreamTestAsync(), StopStreamTestButton);
OpenStreamTestWebButton.Click += (_, _) => _runtime.OpenStreamTestingWebUi();
RepairStreamTestButton.Click += async (_, _) => await RunUiActionAsync(() => _runtime.RepairStreamTestRecoveryAsync(), RepairStreamTestButton);
StartBenchmarkButton.Click += async (_, _) => await RunUiActionAsync(() => _runtime.StartBenchmarkAsync(), StartBenchmarkButton);
StopBenchmarkButton.Click += async (_, _) => await RunUiActionAsync(() => _runtime.StopBenchmarkAsync(), StopBenchmarkButton);
ForgetButton.Click += async (_, _) => await ForgetDeviceAsync();
SaveSettingsButton.Click += async (_, _) => await SaveSettingsAsync();
CheckUpdateButton.Click += async (_, _) => await RunUiActionAsync(() => _runtime.CheckForUpdatesAsync(), CheckUpdateButton);
ApplyUpdateButton.Click += async (_, _) => await ApplyUpdateAsync();
InstallBridgeButton.Click += async (_, _) => await InstallBridgeAsync();
InstallBridgeButton.Click += async (_, _) => await InstallBridgeAsync(InstallBridgeButton);
OverviewUpdateBridgeButton.Click += async (_, _) => await InstallBridgeAsync(OverviewUpdateBridgeButton);
ConnectionUpdateBridgeButton.Click += async (_, _) => await InstallBridgeAsync(ConnectionUpdateBridgeButton);
RemoveBridgeButton.Click += async (_, _) => await RemoveBridgeAsync();
AdvancedToggle.IsCheckedChanged += (_, _) => AdvancedPanel.IsVisible = AdvancedToggle.IsChecked == true;
TranscriptionEnabledToggle.IsCheckedChanged += async (_, _) => await SaveTranscriptionEnabledAsync();
SaveSongOverlayButton.Click += async (_, _) => await SaveSongOverlayAsync();
SaveLumiOverlayButton.Click += async (_, _) => await SaveLumiOverlayAsync();
LumiOverlayEditButton.Click += (_, _) => ToggleLumiOverlayEditMode();
LumiOverlayPreviewButton.Click += (_, _) => { _lumiOverlay.TogglePreview(); RenderLumiOverlay(); };
LumiOverlayVisibilityButton.Click += (_, _) => { _lumiOverlay.ToggleVisibilityOverride(); RenderLumiOverlay(); };
LumiOverlayContainerPicker.SelectionChanged += (_, _) =>
{
if (_renderingLumiOverlay) return;
if (_lumiOverlayDraft is not null)
CaptureLumiOverlayContainer(OverlayContainer(_editingOverlayContainer, _lumiOverlayDraft));
_editingOverlayContainer = (LumiOverlayContainerPicker.SelectedItem as ComboBoxItem)?.Tag?.ToString() ?? "unified";
RenderLumiOverlayContainer();
};
LumiOverlayLayoutPicker.SelectionChanged += (_, _) =>
{
if (_renderingLumiOverlay || _lumiOverlayDraft is null) return;
CaptureLumiOverlayForm(_lumiOverlayDraft);
RenderLumiOverlay(preserveDraft: false);
};
LumiOverlayAnchorPicker.SelectionChanged += (_, _) =>
{
if (_renderingLumiOverlay) return;
UpdateLumiOverlayPaddingAvailability();
};
SendSongOverlayStateButton.Click += async (_, _) => await RunSongOverlayActionAsync(() => _songOverlay.SendSnapshotAsync(), SendSongOverlayStateButton);
ConnectSongOverlaySpotifyButton.Click += async (_, _) =>
{
@ -124,12 +220,19 @@ public partial class MainWindow : Window
public void ShowPage(CompanionPage page)
{
if (page != CompanionPage.LumiOverlay && _lumiOverlay.EditModeActive)
{
_lumiOverlay.EndEdit();
RenderLumiOverlay(preserveDraft: false);
}
var pages = new Dictionary<CompanionPage, Control>
{
[CompanionPage.Overview] = OverviewPage,
[CompanionPage.StreamTesting] = StreamTestingPage,
[CompanionPage.Transcription] = TranscriptionPage,
[CompanionPage.Test] = TestPage,
[CompanionPage.SongOverlay] = SongOverlayPage,
[CompanionPage.LumiOverlay] = LumiOverlayPage,
[CompanionPage.Connection] = ConnectionPage,
[CompanionPage.Logs] = LogsPage,
[CompanionPage.Settings] = SettingsPage
@ -137,13 +240,21 @@ public partial class MainWindow : Window
foreach (var item in pages) item.Value.IsVisible = item.Key == page;
foreach (var item in _navigationButtons)
item.Value.Classes.Set("selected", item.Key == page);
if (_pluginNavigationGroups.TryGetValue(page, out var pluginGroup))
SetPluginNavigationExpanded(pluginGroup.Children, pluginGroup.Arrow, true);
}
private static void SetPluginNavigationExpanded(StackPanel children, TextBlock arrow, bool expanded)
{
children.IsVisible = expanded;
arrow.Text = expanded ? "⌄" : "";
}
private async void OnNextAction(object? sender, RoutedEventArgs args)
{
if (!_runtime.State.Paired) { await ChoosePairingPackageAsync(); return; }
if (!_runtime.State.Connected) { await RunUiActionAsync(() => _runtime.RetryConnectionAsync(), NextActionButton); return; }
if (!_runtime.State.ObsBridgeInstalled) { ShowPage(CompanionPage.Connection); return; }
if (_runtime.State.ObsBridgeUpdateAvailable || !_runtime.State.ObsBridgeInstalled) { ShowPage(CompanionPage.Connection); return; }
ShowPage(CompanionPage.Test);
}
@ -168,9 +279,12 @@ public partial class MainWindow : Window
finally { RenderState(_runtime.State); }
}
private async Task InstallBridgeAsync()
private async Task InstallBridgeAsync(Button initiatingButton)
{
initiatingButton.IsEnabled = false;
InstallBridgeButton.IsEnabled = false;
OverviewUpdateBridgeButton.IsEnabled = false;
ConnectionUpdateBridgeButton.IsEnabled = false;
BridgeStatusText.Text = "Checking the packaged integration and requesting Windows approval…";
try { await _runtime.InstallOrRepairObsBridgeAsync(); }
catch (Exception error)
@ -192,6 +306,10 @@ public partial class MainWindow : Window
{
var settings = _songOverlay.Settings;
settings.Enabled = SongOverlayEnabledToggle.IsChecked == true;
settings.EnabledSources = _songOverlaySourceChecks
.Where(item => item.Value.IsChecked == true)
.Select(item => item.Key)
.ToHashSet(StringComparer.Ordinal);
settings.ProviderId = "spotify";
settings.HeartbeatSeconds = Decimal.ToInt32(SongOverlayHeartbeatBox.Value ?? 30);
settings.SeekThresholdMilliseconds = Decimal.ToInt32(SongOverlaySeekBox.Value ?? 1500);
@ -199,6 +317,7 @@ public partial class MainWindow : Window
settings.UseSearchLinkFallback = SongOverlaySearchLinkToggle.IsChecked == true;
settings.SpotifyClientId = SongOverlaySpotifyClientIdBox.Text?.Trim() ?? "";
await _songOverlay.SaveSettingsAsync(restartProvider);
_songOverlaySourcesDirty = false;
SongOverlayFeedback.Text = "Song Overlay settings saved.";
}
catch (Exception error)
@ -250,6 +369,30 @@ public partial class MainWindow : Window
finally { SaveSettingsButton.IsEnabled = true; }
}
private async Task SaveTranscriptionEnabledAsync()
{
if (_renderingSettings) return;
var enabled = TranscriptionEnabledToggle.IsChecked == true;
TranscriptionEnabledToggle.IsEnabled = false;
TranscriptionEnabledDetail.Text = enabled ? "Enabling speech recognition and caption inclusion…" : "Stopping speech recognition and removing caption inclusion…";
try
{
await _runtime.SetTranscriptionEnabledAsync(enabled);
}
catch (Exception error)
{
TranscriptionEnabledDetail.Text = $"The caption preference could not be changed: {error.Message}";
AddLog($"{DateTime.Now:HH:mm:ss} {error.Message}");
_renderingSettings = true;
TranscriptionEnabledToggle.IsChecked = _settings.Current.TranscriptionEnabled;
_renderingSettings = false;
}
finally
{
TranscriptionEnabledToggle.IsEnabled = !_runtime.State.TestRunning && !_runtime.State.BenchmarkRunning;
}
}
private async Task ForgetDeviceAsync()
{
var dialog = new DecisionWindow("Forget this device?", "This removes the protected credential from this computer. Pairing can be restored with a new package.", "Forget device");
@ -284,9 +427,11 @@ public partial class MainWindow : Window
StatusMark.Background = new SolidColorBrush(Color.Parse(state.Health switch { TrayHealth.Ready => "#176B75", TrayHealth.Operating => "#23845B", TrayHealth.Failed => "#BD4D4D", _ => "#A96612" }));
OverviewDetail.Text = state.Detail;
PairingStepSymbol.Text = state.Paired ? "✓" : "○";
PairingStepDetail.Text = state.Connected ? "Securely connected" : state.Paired ? "Paired, currently offline" : "Not paired";
ObsStepSymbol.Text = state.ObsConnected ? "✓" : state.ObsBridgeInstalled ? "◐" : "○";
ObsStepDetail.Text = state.ObsConnected ? "Connected to OBS" : state.ObsBridgeInstalled ? "Installed; waiting for OBS" : "Installation required";
PairingStepDetail.Text = state.Connected ? "Securely connected" : state.Paired ? "Paired; reconnecting automatically" : "Not paired";
ObsStepSymbol.Text = state.ObsBridgeUpdateAvailable ? "!" : state.ObsConnected ? "✓" : state.ObsBridgeInstalled ? "◐" : "○";
ObsStepDetail.Text = state.ObsBridgeUpdateAvailable
? $"Update required ({state.ObsBridgeLoadedVersion ?? state.ObsBridgeInstalledVersion ?? "unknown"} → {state.ObsBridgeBundledVersion ?? "current"})"
: state.ObsConnected ? "Connected to OBS" : state.ObsBridgeInstalled ? "Installed; waiting for OBS" : "Installation required";
TestStepSymbol.Text = state.PathTestValid ? "✓" : "○";
TestStepDetail.Text = state.PathTestDetail;
DeviceNameText.Text = state.DeviceName ?? "Not paired";
@ -296,21 +441,47 @@ public partial class MainWindow : Window
ReconnectButton.IsEnabled = state.Paired && !state.Connected;
ForgetButton.IsEnabled = state.Paired;
RunTestButton.Content = state.TestRunning ? "Testing…" : "Run full path test";
RunTestButton.IsEnabled = !state.TestRunning && !state.BenchmarkRunning;
StartBenchmarkButton.IsEnabled = !state.TestRunning && !state.BenchmarkRunning;
RunTestButton.IsEnabled = _settings.Current.TranscriptionEnabled && !state.TestRunning && !state.BenchmarkRunning && !state.StreamTestRunning;
StartBenchmarkButton.IsEnabled = _settings.Current.TranscriptionEnabled && !state.TestRunning && !state.BenchmarkRunning && !state.StreamTestRunning;
StopBenchmarkButton.IsEnabled = state.BenchmarkRunning;
TranscriptionEnabledToggle.IsEnabled = !state.TestRunning && !state.BenchmarkRunning;
BenchmarkStatusText.Text = state.BenchmarkDetail;
StreamTestStatusTitle.Text = state.StreamTestRunning
? "PRIVATE TEST ACTIVE"
: state.StreamTestRecoveryRequired
? "OBS restoration required"
: state.StreamTestObsRestartRequired
? "Restart OBS before retrying"
: "Ready to check";
StreamTestStatusDetail.Text = state.StreamTestDetail;
StreamTestBitrateText.Text = state.StreamTestRunning ? $"{state.StreamTestBitrateKbps:0} kbps" : "—";
StreamTestDroppedText.Text = state.StreamTestRunning ? $"{state.StreamTestDroppedFrames} / {state.StreamTestTotalFrames}" : "—";
StreamTestCongestionText.Text = state.StreamTestRunning ? $"{state.StreamTestCongestion:P0}" : "—";
StartStreamTestButton.IsEnabled = state.Connected && state.ObsConnected && !state.ObsBridgeUpdateAvailable && !state.ObsBridgeRepairNeeded && !state.ObsStreaming && !state.ObsRecording && !state.TestRunning && !state.BenchmarkRunning && !state.StreamTestRunning && !state.StreamTestRecoveryRequired && !state.StreamTestObsRestartRequired;
StopStreamTestButton.IsEnabled = state.StreamTestRunning;
OpenStreamTestWebButton.IsVisible = state.StreamTestRunning;
OpenStreamTestWebButton.IsEnabled = state.StreamTestRunning && state.Host is not null;
RepairStreamTestButton.IsVisible = state.StreamTestRecoveryRequired && !state.StreamTestRunning;
UpdatePanel.IsVisible = state.UpdateAvailable;
UpdateTitle.Text = state.UpdateAvailable ? $"Lumi Companion {state.AvailableVersion} is available" : "Lumi Companion is current";
UpdateDetail.Text = state.UpdateDetail;
CheckUpdateButton.IsEnabled = !state.UpdateCheckRunning;
CheckUpdateButton.Content = state.UpdateCheckRunning ? "Checking…" : "Check now";
ApplyUpdateButton.IsEnabled = state.UpdateAvailable && !state.ObsStreaming && !state.ObsRecording;
CurrentVersionText.Text = CompanionRuntime.DisplayVersion;
UpdateStatusText.Text = state.UpdateDetail;
BridgeStatusText.Text = state.ObsBridgeDetail;
InstallBridgeButton.Content = state.ObsBridgeRepairNeeded ? "Repair integration" : state.ObsBridgeInstalled ? "Reinstall integration" : "Install integration";
OverviewBridgeUpdatePanel.IsVisible = state.ObsBridgeUpdateAvailable;
ConnectionBridgeUpdatePanel.IsVisible = state.ObsBridgeUpdateAvailable;
OverviewBridgeUpdateDetail.Text = state.ObsBridgeDetail;
ConnectionBridgeUpdateDetail.Text = state.ObsBridgeDetail;
InstallBridgeButton.Content = state.ObsBridgeUpdateAvailable ? "Update OBS Bridge" : state.ObsBridgeRepairNeeded ? "Repair integration" : state.ObsBridgeInstalled ? "Reinstall integration" : "Install integration";
// The maintenance action performs a fresh package/process check and reports the
// actual blocker, so a stale OBS connection signal must never strand repair.
InstallBridgeButton.IsEnabled = !state.TestRunning && !state.BenchmarkRunning;
var bridgeMaintenanceEnabled = !state.TestRunning && !state.BenchmarkRunning && !state.StreamTestRunning;
InstallBridgeButton.IsEnabled = bridgeMaintenanceEnabled;
OverviewUpdateBridgeButton.IsEnabled = bridgeMaintenanceEnabled;
ConnectionUpdateBridgeButton.IsEnabled = bridgeMaintenanceEnabled;
RemoveBridgeButton.IsEnabled = state.ObsBridgeInstalled || state.ObsBridgeRepairNeeded;
if (!state.Paired)
@ -321,9 +492,15 @@ public partial class MainWindow : Window
}
else if (!state.Connected)
{
NextActionTitle.Text = "Reconnect to Lumi";
NextActionDetail.Text = "Your device is paired, but Lumi cannot currently be reached.";
NextActionButton.Content = "Retry connection";
NextActionTitle.Text = "Lumi is temporarily unavailable";
NextActionDetail.Text = "Companion is reconnecting automatically in the background.";
NextActionButton.Content = "Retry now";
}
else if (state.ObsBridgeUpdateAvailable)
{
NextActionTitle.Text = "Update OBS Bridge";
NextActionDetail.Text = state.ObsBridgeDetail;
NextActionButton.Content = "Manage OBS integration";
}
else if (!state.ObsBridgeInstalled)
{
@ -341,15 +518,376 @@ public partial class MainWindow : Window
private void RenderSettings(CompanionSettings settings)
{
AutoStartToggle.IsChecked = settings.AutoStartWithWindows;
StartWithObsToggle.IsChecked = settings.StartWithObs;
AdvancedToggle.IsChecked = settings.AdvancedMode;
AdvancedPanel.IsVisible = settings.AdvancedMode;
if (settings.PrimarySourceUuid is not null)
SourcePicker.SelectedItem = _runtime.ObsSources.FirstOrDefault(source => source.Uuid == settings.PrimarySourceUuid);
_renderingSettings = true;
try
{
AutoStartToggle.IsChecked = settings.AutoStartWithWindows;
StartWithObsToggle.IsChecked = settings.StartWithObs;
TranscriptionEnabledToggle.IsChecked = settings.TranscriptionEnabled;
TranscriptionEnabledDetail.Text = settings.TranscriptionEnabled
? "Speech recognition and caption inclusion are enabled."
: "Captions are off. Companion will not send microphone audio for speech recognition.";
AdvancedToggle.IsChecked = settings.AdvancedMode;
AdvancedPanel.IsVisible = settings.AdvancedMode;
if (settings.PrimarySourceUuid is not null)
SourcePicker.SelectedItem = _runtime.ObsSources.FirstOrDefault(source => source.Uuid == settings.PrimarySourceUuid);
}
finally
{
_renderingSettings = false;
}
}
private void PrepareLumiOverlayControls()
{
LumiOverlayVisibilityPicker.ItemsSource = new[]
{
new OverlayChoice<OverlayVisibilityMode>(OverlayVisibilityMode.Off, "Off"),
new OverlayChoice<OverlayVisibilityMode>(OverlayVisibilityMode.WhenLive, "When a selected channel is live"),
new OverlayChoice<OverlayVisibilityMode>(OverlayVisibilityMode.On, "Always on")
};
LumiOverlayLayoutPicker.ItemsSource = new[]
{
new OverlayChoice<OverlayLayoutMode>(OverlayLayoutMode.Unified, "Unified chat and events"),
new OverlayChoice<OverlayLayoutMode>(OverlayLayoutMode.DividedHorizontal, "Divided horizontally"),
new OverlayChoice<OverlayLayoutMode>(OverlayLayoutMode.DividedVertical, "Divided vertically"),
new OverlayChoice<OverlayLayoutMode>(OverlayLayoutMode.Separate, "Separate containers")
};
LumiOverlayAnchorPicker.ItemsSource = new[]
{
new OverlayChoice<OverlayAnchor>(OverlayAnchor.TopLeft, "Top left"),
new OverlayChoice<OverlayAnchor>(OverlayAnchor.Top, "Top"),
new OverlayChoice<OverlayAnchor>(OverlayAnchor.TopRight, "Top right"),
new OverlayChoice<OverlayAnchor>(OverlayAnchor.Left, "Left"),
new OverlayChoice<OverlayAnchor>(OverlayAnchor.Right, "Right"),
new OverlayChoice<OverlayAnchor>(OverlayAnchor.BottomLeft, "Bottom left"),
new OverlayChoice<OverlayAnchor>(OverlayAnchor.Bottom, "Bottom"),
new OverlayChoice<OverlayAnchor>(OverlayAnchor.BottomRight, "Bottom right")
};
LumiOverlayDirectionPicker.ItemsSource = new[]
{
new OverlayChoice<OverlayNewestDirection>(OverlayNewestDirection.Start, "New items first"),
new OverlayChoice<OverlayNewestDirection>(OverlayNewestDirection.End, "New items last")
};
LumiOverlayHorizontalAlignPicker.ItemsSource = new[]
{
new OverlayChoice<OverlayHorizontalAlignment>(OverlayHorizontalAlignment.Left, "Left"),
new OverlayChoice<OverlayHorizontalAlignment>(OverlayHorizontalAlignment.Middle, "Center"),
new OverlayChoice<OverlayHorizontalAlignment>(OverlayHorizontalAlignment.Right, "Right")
};
LumiOverlayVerticalAlignPicker.ItemsSource = new[]
{
new OverlayChoice<OverlayVerticalAlignment>(OverlayVerticalAlignment.Top, "Top"),
new OverlayChoice<OverlayVerticalAlignment>(OverlayVerticalAlignment.Middle, "Center"),
new OverlayChoice<OverlayVerticalAlignment>(OverlayVerticalAlignment.Bottom, "Bottom")
};
var animations = new[]
{
new OverlayChoice<OverlayAnimation>(OverlayAnimation.None, "None"),
new OverlayChoice<OverlayAnimation>(OverlayAnimation.Fade, "Fade"),
new OverlayChoice<OverlayAnimation>(OverlayAnimation.Slide, "Slide"),
new OverlayChoice<OverlayAnimation>(OverlayAnimation.FadeSlide, "Fade + slide"),
new OverlayChoice<OverlayAnimation>(OverlayAnimation.ScaleFade, "Scale + fade")
};
LumiOverlayEntryPicker.ItemsSource = animations;
LumiOverlayExitPicker.ItemsSource = animations;
LumiOverlayEasingPicker.ItemsSource = new[] { "linear", "cubic-out", "cubic-in-out" };
LumiOverlayFontBox.ItemsSource = FontManager.Current.SystemFonts.Select(font => font.Name)
.Distinct(StringComparer.OrdinalIgnoreCase).OrderBy(name => name, StringComparer.OrdinalIgnoreCase).ToList();
LumiOverlayContainerPicker.SelectedIndex = 0;
}
private void RenderLumiOverlay(bool preserveDraft = true)
{
if (!_lumiOverlay.IsInitialized) return;
if (_lumiOverlayDraft is null)
_lumiOverlayDraft = _lumiOverlay.Settings.CreateCopy();
else if (preserveDraft && !_renderingLumiOverlay && !_lumiOverlay.EditModeActive)
CaptureLumiOverlayForm(_lumiOverlayDraft);
_renderingLumiOverlay = true;
try
{
var settings = _lumiOverlayDraft;
LumiOverlayFeedStatus.Text = _lumiOverlay.Status.Summary;
LumiOverlayFeedDetail.Text = _lumiOverlay.Status.Detail ?? _lumiOverlay.Status.Summary;
LumiOverlayProviderStatus.Text = _lumiOverlay.FeedStatuses.Count == 0
? "Provider status will appear after Lumi connects."
: string.Join(" · ", _lumiOverlay.FeedStatuses.Select(status => $"{status.Platform}: {status.Detail}"));
LumiOverlayCaptureStatus.Text = _lumiOverlay.CaptureExclusion.State.ToString();
LumiOverlayCaptureDetail.Text = _lumiOverlay.CaptureExclusion.Detail;
LumiOverlayMonitorWarning.Text = _lumiOverlay.MonitorWarning ?? "";
LumiOverlayMonitorWarning.IsVisible = _lumiOverlay.MonitorWarning is not null;
LumiOverlayEnabledToggle.IsChecked = settings.Enabled;
SelectOverlayChoice(LumiOverlayVisibilityPicker, settings.Visibility);
SelectOverlayChoice(LumiOverlayLayoutPicker, settings.Layout);
LumiOverlayChatPercentageBox.Value = settings.ChatPercentage;
UpdateLumiOverlayLayoutPresentation(settings.Layout);
LumiOverlayPreviewButton.Content = _lumiOverlay.PreviewActive ? "End sample preview" : "Start sample preview";
var visibility = _lumiOverlay.VisibilityPresentation;
LumiOverlayVisibilityState.Text = $"Current display: {visibility.State}";
LumiOverlayVisibilityButton.Content = visibility.Action;
LumiOverlayVisibilityButton.IsEnabled = visibility.CanToggle;
_visibilityHotkeyCapture.Hotkey = settings.ToggleVisibilityHotkey;
_previewHotkeyCapture.Hotkey = settings.TogglePreviewHotkey;
LumiOverlayHotkeyStatus.Text = _lumiOverlay.HotkeyStatus;
LumiOverlayEditButton.Content = _lumiOverlay.EditModeActive ? "End edit mode" : "Start edit mode";
LumiOverlayEditStatus.Text = _lumiOverlay.EditModeActive
? "Edit mode is active on the selected monitor. Drag to move; use the lower-right corner to resize."
: "Edit mode is off. Changes remain a draft until saved.";
var monitorItems = _lumiOverlay.Monitors.Select(value => new OverlayPickerItem(value.Id, value.Label)).ToList();
if (settings.MonitorId is { Length: > 0 } savedMonitor &&
monitorItems.All(value => !string.Equals(value.Id, savedMonitor, StringComparison.Ordinal)))
monitorItems.Insert(0, new OverlayPickerItem(savedMonitor, "Previously selected monitor · unavailable"));
LumiOverlayMonitorPicker.ItemsSource = monitorItems;
LumiOverlayMonitorPicker.SelectedItem = monitorItems.FirstOrDefault(value => value.Id == settings.MonitorId) ?? monitorItems.FirstOrDefault();
_overlaySourceChecks.Clear();
LumiOverlaySourcesPanel.Children.Clear();
var available = _lumiOverlay.Sources.ToDictionary(source => source.Id, StringComparer.Ordinal);
var merged = settings.Sources.Select(source =>
new OverlaySourceView(source, available.GetValueOrDefault(source.Id))).ToList();
foreach (var source in _lumiOverlay.Sources.Where(source => settings.Sources.All(saved => saved.Id != source.Id)))
merged.Add(new(new OverlaySourceSetting { Id = source.Id, Platform = source.Platform, Label = source.Label }, source));
foreach (var entry in merged.OrderBy(value => value.Saved.Label, StringComparer.OrdinalIgnoreCase))
{
var check = new CheckBox
{
Content = entry.Current is null ? $"{entry.Saved.Label} · unavailable" : $"{entry.Current.Label} · {entry.Current.Status}",
IsChecked = entry.Saved.Selected
};
LumiOverlaySourcesPanel.Children.Add(check);
_overlaySourceChecks[entry.Saved.Id] = check;
}
if (_overlaySourceChecks.Count == 0) LumiOverlaySourcesPanel.Children.Add(new TextBlock { Text = "No configured provider sources are available.", Classes = { "muted" } });
_overlayEventChecks.Clear();
LumiOverlayEventsPanel.Children.Clear();
var availableEvents = _lumiOverlay.AvailableEventTypes.ToDictionary(value => value.Id, StringComparer.Ordinal);
foreach (var eventType in availableEvents.Values.OrderBy(value => value.Platform).ThenBy(value => value.Label))
{
var check = new CheckBox { Content = $"{eventType.Label} · {eventType.Platform}", IsChecked = settings.EventTypes.Contains(eventType.Id) };
LumiOverlayEventsPanel.Children.Add(check);
_overlayEventChecks[eventType.Id] = check;
}
foreach (var eventId in settings.EventTypes.Where(id => !availableEvents.ContainsKey(id)).OrderBy(id => id))
{
var check = new CheckBox { Content = $"{eventId} · unavailable", IsChecked = true };
LumiOverlayEventsPanel.Children.Add(check);
_overlayEventChecks[eventId] = check;
}
if (_overlayEventChecks.Count == 0) LumiOverlayEventsPanel.Children.Add(new TextBlock { Text = "No event types were reported by Lumi.", Classes = { "muted" } });
RenderLumiOverlayContainer();
}
finally { _renderingLumiOverlay = false; }
}
private OverlayContainerSettings SelectedOverlayContainer()
{
var tag = (LumiOverlayContainerPicker.SelectedItem as ComboBoxItem)?.Tag?.ToString();
return OverlayContainer(tag, _lumiOverlayDraft ?? _lumiOverlay.Settings);
}
private static OverlayContainerSettings OverlayContainer(string? tag, LumiOverlaySettings settings) => tag switch
{
"chat" => settings.Chat,
"events" => settings.Events,
_ => settings.Unified
};
private void RenderLumiOverlayContainer()
{
var value = SelectedOverlayContainer();
SelectOverlayChoice(LumiOverlayAnchorPicker, value.Anchor);
SelectOverlayChoice(LumiOverlayDirectionPicker, value.NewestDirection);
SelectOverlayChoice(LumiOverlayHorizontalAlignPicker, value.HorizontalAlignment);
SelectOverlayChoice(LumiOverlayVerticalAlignPicker, value.VerticalAlignment);
LumiOverlayWidthBox.Value = value.Width;
LumiOverlayHeightBox.Value = value.Height;
LumiOverlayCardColorBox.Text = value.CardBackground;
LumiOverlayTextColorBox.Text = value.TextColor;
LumiOverlayFontBox.SelectedItem = value.FontFamily;
LumiOverlayFontSizeBox.Value = Convert.ToDecimal(value.FontSize);
SelectOverlayChoice(LumiOverlayEntryPicker, value.EntryAnimation);
SelectOverlayChoice(LumiOverlayExitPicker, value.ExitAnimation);
LumiOverlayLifetimeBox.Value = value.LifetimeMs;
LumiOverlayQueueBox.Value = value.QueueLimit;
LumiOverlayPaddingTopBox.Value = value.PaddingTop;
LumiOverlayPaddingRightBox.Value = value.PaddingRight;
LumiOverlayPaddingBottomBox.Value = value.PaddingBottom;
LumiOverlayPaddingLeftBox.Value = value.PaddingLeft;
LumiOverlayShadowColorBox.Text = value.TextShadow;
LumiOverlayShadowXBox.Value = value.ShadowX;
LumiOverlayShadowYBox.Value = value.ShadowY;
LumiOverlayShadowBlurBox.Value = value.ShadowBlur;
LumiOverlayEntryDurationBox.Value = value.EntryDurationMs;
LumiOverlayExitDurationBox.Value = value.ExitDurationMs;
LumiOverlaySlideDistanceBox.Value = value.SlideDistance;
LumiOverlayScaleAmountBox.Value = Convert.ToDecimal(value.ScaleAmount);
LumiOverlayEasingPicker.SelectedItem = value.Easing;
UpdateLumiOverlayPaddingAvailability();
LumiOverlayAvatarToggle.IsChecked = value.ShowAvatar;
LumiOverlayUsernameToggle.IsChecked = value.ShowUsername;
LumiOverlayPlatformToggle.IsChecked = value.ShowPlatform;
}
private void CaptureLumiOverlayContainer(OverlayContainerSettings? target = null)
{
var value = target ?? SelectedOverlayContainer();
if (LumiOverlayAnchorPicker.SelectedItem is OverlayChoice<OverlayAnchor> anchor) value.Anchor = anchor.Value;
if (LumiOverlayDirectionPicker.SelectedItem is OverlayChoice<OverlayNewestDirection> direction) value.NewestDirection = direction.Value;
if (LumiOverlayHorizontalAlignPicker.SelectedItem is OverlayChoice<OverlayHorizontalAlignment> horizontal) value.HorizontalAlignment = horizontal.Value;
if (LumiOverlayVerticalAlignPicker.SelectedItem is OverlayChoice<OverlayVerticalAlignment> vertical) value.VerticalAlignment = vertical.Value;
if (LumiOverlayEntryPicker.SelectedItem is OverlayChoice<OverlayAnimation> entry) value.EntryAnimation = entry.Value;
if (LumiOverlayExitPicker.SelectedItem is OverlayChoice<OverlayAnimation> exit) value.ExitAnimation = exit.Value;
value.Width = Decimal.ToInt32(LumiOverlayWidthBox.Value ?? value.Width);
value.Height = Decimal.ToInt32(LumiOverlayHeightBox.Value ?? value.Height);
value.CardBackground = LumiOverlayCardColorBox.Text?.Trim() ?? value.CardBackground;
value.TextColor = LumiOverlayTextColorBox.Text?.Trim() ?? value.TextColor;
value.FontFamily = LumiOverlayFontBox.SelectedItem?.ToString()?.Trim() ?? value.FontFamily;
value.FontSize = Convert.ToDouble(LumiOverlayFontSizeBox.Value ?? Convert.ToDecimal(value.FontSize));
value.LifetimeMs = Decimal.ToInt32(LumiOverlayLifetimeBox.Value ?? value.LifetimeMs);
value.QueueLimit = Decimal.ToInt32(LumiOverlayQueueBox.Value ?? value.QueueLimit);
value.PaddingTop = Decimal.ToInt32(LumiOverlayPaddingTopBox.Value ?? value.PaddingTop);
value.PaddingRight = Decimal.ToInt32(LumiOverlayPaddingRightBox.Value ?? value.PaddingRight);
value.PaddingBottom = Decimal.ToInt32(LumiOverlayPaddingBottomBox.Value ?? value.PaddingBottom);
value.PaddingLeft = Decimal.ToInt32(LumiOverlayPaddingLeftBox.Value ?? value.PaddingLeft);
value.TextShadow = LumiOverlayShadowColorBox.Text?.Trim() ?? value.TextShadow;
value.ShadowX = Decimal.ToInt32(LumiOverlayShadowXBox.Value ?? value.ShadowX);
value.ShadowY = Decimal.ToInt32(LumiOverlayShadowYBox.Value ?? value.ShadowY);
value.ShadowBlur = Decimal.ToInt32(LumiOverlayShadowBlurBox.Value ?? value.ShadowBlur);
value.EntryDurationMs = Decimal.ToInt32(LumiOverlayEntryDurationBox.Value ?? value.EntryDurationMs);
value.ExitDurationMs = Decimal.ToInt32(LumiOverlayExitDurationBox.Value ?? value.ExitDurationMs);
value.SlideDistance = Decimal.ToInt32(LumiOverlaySlideDistanceBox.Value ?? value.SlideDistance);
value.ScaleAmount = Convert.ToDouble(LumiOverlayScaleAmountBox.Value ?? Convert.ToDecimal(value.ScaleAmount));
value.Easing = LumiOverlayEasingPicker.SelectedItem?.ToString() ?? value.Easing;
value.ShowAvatar = LumiOverlayAvatarToggle.IsChecked == true;
value.ShowUsername = LumiOverlayUsernameToggle.IsChecked == true;
value.ShowPlatform = LumiOverlayPlatformToggle.IsChecked == true;
}
private void CaptureLumiOverlayForm(LumiOverlaySettings settings)
{
CaptureLumiOverlayContainer(OverlayContainer(_editingOverlayContainer, settings));
settings.Enabled = LumiOverlayEnabledToggle.IsChecked == true;
if (LumiOverlayVisibilityPicker.SelectedItem is OverlayChoice<OverlayVisibilityMode> visibility)
settings.Visibility = visibility.Value;
if (LumiOverlayLayoutPicker.SelectedItem is OverlayChoice<OverlayLayoutMode> layout)
settings.Layout = layout.Value;
settings.ChatPercentage = Decimal.ToInt32(LumiOverlayChatPercentageBox.Value ?? settings.ChatPercentage);
if (LumiOverlayMonitorPicker.SelectedItem is OverlayPickerItem monitor) settings.MonitorId = monitor.Id;
settings.ToggleVisibilityHotkey = _visibilityHotkeyCapture.Hotkey;
settings.TogglePreviewHotkey = _previewHotkeyCapture.Hotkey;
var currentSources = _lumiOverlay.Sources.ToDictionary(source => source.Id, StringComparer.Ordinal);
foreach (var saved in settings.Sources)
{
if (_overlaySourceChecks.TryGetValue(saved.Id, out var check)) saved.Selected = check.IsChecked == true;
if (currentSources.TryGetValue(saved.Id, out var current))
{
saved.Platform = current.Platform;
saved.Label = current.Label;
}
}
foreach (var source in currentSources.Values.Where(source => settings.Sources.All(saved => saved.Id != source.Id)))
settings.Sources.Add(new OverlaySourceSetting
{
Id = source.Id,
Platform = source.Platform,
Label = source.Label,
Selected = _overlaySourceChecks.GetValueOrDefault(source.Id)?.IsChecked == true
});
settings.EventTypes = _overlayEventChecks
.Where(item => item.Value.IsChecked == true)
.Select(item => item.Key)
.ToHashSet(StringComparer.Ordinal);
}
private void UpdateLumiOverlayLayoutPresentation(OverlayLayoutMode layout)
{
var divided = layout is OverlayLayoutMode.DividedHorizontal or OverlayLayoutMode.DividedVertical;
LumiOverlayChatShareLabel.IsVisible = divided;
LumiOverlayChatShareEditor.IsVisible = divided;
var unified = layout == OverlayLayoutMode.Unified;
LumiOverlayUnifiedContainerOption.IsVisible = unified;
LumiOverlayChatContainerOption.IsVisible = !unified;
LumiOverlayEventContainerOption.IsVisible = !unified;
var wanted = unified ? "unified" : _editingOverlayContainer == "unified" ? "chat" : _editingOverlayContainer;
var selected = LumiOverlayContainerPicker.Items
.OfType<ComboBoxItem>()
.First(item => string.Equals(item.Tag?.ToString(), wanted, StringComparison.Ordinal));
_editingOverlayContainer = wanted;
LumiOverlayContainerPicker.SelectedItem = selected;
}
private void UpdateLumiOverlayPaddingAvailability()
{
var anchor = LumiOverlayAnchorPicker.SelectedItem is OverlayChoice<OverlayAnchor> choice
? choice.Value
: OverlayAnchor.BottomLeft;
LumiOverlayPaddingTopBox.IsEnabled =
anchor is OverlayAnchor.TopLeft or OverlayAnchor.Top or OverlayAnchor.TopRight or OverlayAnchor.Left or OverlayAnchor.Right;
LumiOverlayPaddingRightBox.IsEnabled =
anchor is OverlayAnchor.TopRight or OverlayAnchor.Right or OverlayAnchor.BottomRight or OverlayAnchor.Top or OverlayAnchor.Bottom;
LumiOverlayPaddingBottomBox.IsEnabled =
anchor is OverlayAnchor.BottomLeft or OverlayAnchor.Bottom or OverlayAnchor.BottomRight or OverlayAnchor.Left or OverlayAnchor.Right;
LumiOverlayPaddingLeftBox.IsEnabled =
anchor is OverlayAnchor.TopLeft or OverlayAnchor.Left or OverlayAnchor.BottomLeft or OverlayAnchor.Top or OverlayAnchor.Bottom;
}
private static void SelectOverlayChoice<T>(ComboBox picker, T value) where T : struct, Enum
{
picker.SelectedItem = picker.ItemsSource?.OfType<OverlayChoice<T>>().FirstOrDefault(item => EqualityComparer<T>.Default.Equals(item.Value, value));
}
private void ToggleLumiOverlayEditMode()
{
if (_lumiOverlayDraft is null) _lumiOverlayDraft = _lumiOverlay.Settings.CreateCopy();
if (_lumiOverlay.EditModeActive)
{
_lumiOverlay.EndEdit();
RenderLumiOverlay(preserveDraft: false);
return;
}
CaptureLumiOverlayForm(_lumiOverlayDraft);
_lumiOverlay.BeginEdit(_lumiOverlayDraft);
RenderLumiOverlay(preserveDraft: false);
}
private async Task SaveLumiOverlayAsync()
{
if (_renderingLumiOverlay) return;
SaveLumiOverlayButton.IsEnabled = false;
LumiOverlayFeedback.Text = "Saving…";
try
{
var settings = _lumiOverlayDraft ?? _lumiOverlay.Settings.CreateCopy();
if (_lumiOverlay.EditModeActive)
{
_lumiOverlay.EndEdit();
RenderLumiOverlay(preserveDraft: false);
}
CaptureLumiOverlayForm(settings);
await _lumiOverlay.SaveSettingsAsync(settings);
_lumiOverlayDraft = _lumiOverlay.Settings.CreateCopy();
LumiOverlayFeedback.Text = "Lumi Overlay settings saved and applied.";
}
catch (Exception error) { LumiOverlayFeedback.Text = $"Could not save Lumi Overlay: {error.Message}"; }
finally { SaveLumiOverlayButton.IsEnabled = true; RenderLumiOverlay(); }
}
private sealed record OverlayPickerItem(string Id, string Label)
{
public override string ToString() => Label;
}
private sealed record OverlayChoice<T>(T Value, string Label) where T : struct, Enum
{
public override string ToString() => Label;
}
private sealed record OverlaySourceView(OverlaySourceSetting Saved, OverlaySource? Current);
private void RenderSongOverlay()
{
_renderingSongOverlay = true;
@ -367,6 +905,30 @@ public partial class MainWindow : Window
SongOverlaySeekBox.Value = Math.Clamp(settings.SeekThresholdMilliseconds, 500, 10000);
SongOverlayCoverToggle.IsChecked = settings.SendCoverArt;
SongOverlaySearchLinkToggle.IsChecked = settings.UseSearchLinkFallback;
if (_songOverlaySourceChecks.Count == 0)
{
foreach (var source in MediaSourceCatalog.All)
{
var check = new CheckBox
{
Content = source.Label,
IsChecked = settings.EnabledSources.Contains(source.Id)
};
ToolTip.SetTip(check, source.Detail);
check.IsCheckedChanged += (_, _) =>
{
if (!_renderingSongOverlay) _songOverlaySourcesDirty = true;
};
SongOverlaySourcesPanel.Children.Add(check);
_songOverlaySourceChecks[source.Id] = check;
}
}
else if (!_songOverlaySourcesDirty)
{
foreach (var source in MediaSourceCatalog.All)
if (_songOverlaySourceChecks.TryGetValue(source.Id, out var check))
check.IsChecked = settings.EnabledSources.Contains(source.Id);
}
if (!SongOverlaySpotifyClientIdBox.IsFocused) SongOverlaySpotifyClientIdBox.Text = settings.SpotifyClientId;
SongOverlaySpotifyStatus.Text = _songOverlay.IsSpotifyEnrichmentConnected
? "Connected. Exact links, release year and official artwork can be enriched on song changes."
@ -406,7 +968,10 @@ public partial class MainWindow : Window
TestStepSymbol.Text = valid ? "✓" : "○";
TestStepDetail.Text = currentPass ? "Passed just now; this result will be reused while the relevant setup stays unchanged." : _runtime.State.PathTestDetail;
RunTestButton.Content = _runtime.State.TestRunning ? "Testing…" : "Run full path test";
RunTestButton.IsEnabled = !_runtime.State.TestRunning && !_runtime.State.BenchmarkRunning;
RunTestButton.IsEnabled = _settings.Current.TranscriptionEnabled &&
!_runtime.State.TestRunning &&
!_runtime.State.BenchmarkRunning &&
!_runtime.State.StreamTestRunning;
}
private void RenderBenchmark(BenchmarkSnapshot benchmark)
@ -426,6 +991,8 @@ public partial class MainWindow : Window
VoiceLevelMeter.Foreground = new SolidColorBrush(Color.Parse(color));
}
private sealed record PluginNavigationGroup(StackPanel Children, TextBlock Arrow);
private static void RenderMetric(Panel panel, MetricStatistics metric, Func<double, string> formatter)
{
panel.Children.Clear();
@ -454,6 +1021,7 @@ public partial class MainWindow : Window
private void OnClosing(object? sender, WindowClosingEventArgs args)
{
if (_allowExit) return;
if (_lumiOverlay.EditModeActive) _lumiOverlay.EndEdit();
args.Cancel = true;
Hide();
}
@ -467,7 +1035,7 @@ public partial class MainWindow : Window
public void AllowExit() => _allowExit = true;
private static (CompanionRuntime, CompanionSettingsStore, SongOverlayRuntime, IReadOnlyList<ICompanionPluginContribution>) CreateDefaultServices()
private static (CompanionRuntime, CompanionSettingsStore, SongOverlayRuntime, LumiOverlayRuntime, IReadOnlyList<ICompanionPluginContribution>) CreateDefaultServices()
{
var paths = CompanionPaths.ForCurrentUser();
var settings = new CompanionSettingsStore(paths.SettingsPath);
@ -476,7 +1044,10 @@ public partial class MainWindow : Window
Path.Combine(paths.Root, "plugins", "now_playing"),
paths.LogsDirectory,
runtime.CreatePluginTransport("now_playing"));
ICompanionPluginContribution[] plugins = [new TranscriptionPluginContribution(runtime), songOverlay];
return (runtime, settings, songOverlay, plugins);
var lumiOverlay = new LumiOverlayRuntime(
Path.Combine(paths.Root, "plugins", "lumi_overlay"),
runtime.CreatePluginTransport("lumi_overlay"));
ICompanionPluginContribution[] plugins = [new TranscriptionPluginContribution(runtime), lumiOverlay, songOverlay];
return (runtime, settings, songOverlay, lumiOverlay, plugins);
}
}

View File

@ -23,9 +23,11 @@ public sealed class ObsBridgeManager
var packageAvailable = package is not null;
var fileInstalled = File.Exists(InstalledDll);
var valid = packageAvailable && fileInstalled && HashFile(InstalledDll) == package!.Manifest.Sha256 && installed?.Version == package.Manifest.Version;
return new ObsBridgeStatus(packageAvailable, fileInstalled, valid, package?.Manifest.Version,
var updateAvailable = packageAvailable && fileInstalled && installed is not null && installed.Version != package!.Manifest.Version;
return new ObsBridgeStatus(packageAvailable, fileInstalled, valid, updateAvailable, installed?.Version, package?.Manifest.Version,
valid ? $"OBS integration {package!.Manifest.Version} is installed. Restart OBS if it was open during the last repair." :
!packageAvailable ? $"The bundled OBS integration could not be verified. {_packageFailure ?? "The packaged component was not found."}" :
updateAvailable ? $"OBS Bridge {installed!.Version} is installed; Companion includes {package!.Manifest.Version}. Close OBS, then update the bridge." :
fileInstalled ? "The OBS integration is outdated or damaged. Repair it while OBS is closed." : "The OBS integration is ready to install.");
}
@ -242,5 +244,12 @@ public sealed record ObsBridgeManifest(
[property: JsonPropertyName("version")] string Version,
[property: JsonPropertyName("sha256")] string Sha256,
[property: JsonPropertyName("obs_minimum_version")] string ObsMinimumVersion);
public sealed record ObsBridgeStatus(bool PackageAvailable, bool Installed, bool Valid, string? Version, string Detail);
public sealed record ObsBridgeStatus(
bool PackageAvailable,
bool Installed,
bool Valid,
bool UpdateAvailable,
string? InstalledVersion,
string? Version,
string Detail);
internal sealed record ObsBridgePackage(ObsBridgeManifest Manifest, byte[] Dll, byte[]? Locale);

View File

@ -11,11 +11,21 @@ internal sealed class TranscriptionPluginContribution : ICompanionPluginContribu
_runtime = runtime;
Actions =
[
new CompanionPluginAction(
"toggle",
() => _runtime.TranscriptionEnabled ? "Turn captions off" : "Turn captions on",
async cancellationToken =>
{
await _runtime.SetTranscriptionEnabledAsync(!_runtime.TranscriptionEnabled, cancellationToken);
Changed?.Invoke();
},
() => !_runtime.State.TestRunning && !_runtime.State.BenchmarkRunning,
5),
new CompanionPluginAction(
"test",
() => _runtime.State.TestRunning || _runtime.State.BenchmarkRunning ? "Transcription test running…" : "Run transcription test",
cancellationToken => _runtime.RunTestAsync(cancellationToken),
() => !_runtime.State.TestRunning && !_runtime.State.BenchmarkRunning,
() => _runtime.TranscriptionEnabled && !_runtime.State.TestRunning && !_runtime.State.BenchmarkRunning,
10)
];
_runtime.StateChanged += _ => Changed?.Invoke();
@ -37,15 +47,17 @@ internal sealed class TranscriptionPluginContribution : ICompanionPluginContribu
public IReadOnlyList<CompanionPluginAction> Actions { get; }
public CompanionPluginStatus Status => new(
_runtime.State.Health switch
!_runtime.TranscriptionEnabled
? CompanionPluginHealth.Healthy
: _runtime.State.Health switch
{
TrayHealth.Ready => CompanionPluginHealth.Healthy,
TrayHealth.Operating => CompanionPluginHealth.Healthy,
TrayHealth.Failed => CompanionPluginHealth.Error,
_ => CompanionPluginHealth.Warning
},
_runtime.State.Summary,
_runtime.State.Detail);
_runtime.TranscriptionEnabled ? _runtime.State.Summary : "Captions off",
_runtime.TranscriptionEnabled ? _runtime.State.Detail : "Microphone audio is not sent for speech recognition, and captions are not included.");
public event Action? Changed;
}

View File

@ -1,5 +1,9 @@
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Net.WebSockets;
using System.Runtime.CompilerServices;
using System.Text;
using System.Text.Json;
using Lumi.Companion.Abstractions;
using Lumi.Companion.Protocol;
@ -8,7 +12,8 @@ namespace Lumi.Companion.Core;
public sealed class CompanionPluginTransport(
HttpClient http,
Func<DeviceCredential?> credentialProvider,
string pluginId) : ICompanionPluginTransport
string pluginId,
Action<string, string>? log = null) : ICompanionPluginTransport
{
private readonly string _pluginId = ValidatePluginId(pluginId);
@ -65,6 +70,113 @@ public sealed class CompanionPluginTransport(
return new CompanionPluginHttpResponse((int)response.StatusCode, body);
}
public async Task RunLiveJsonAsync(
string relativePath,
Func<ICompanionPluginLiveChannel, CancellationToken, Task> connected,
Action<CompanionPluginLiveStatus>? status,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(connected);
var attempt = 0;
while (!cancellationToken.IsCancellationRequested)
{
var credential = credentialProvider();
if (credential is null)
{
status?.Invoke(new(CompanionPluginLiveState.WaitingForPairing, "Pair Companion with Lumi to use this plugin.", attempt));
await Task.Delay(TimeSpan.FromSeconds(5), cancellationToken).ConfigureAwait(false);
continue;
}
try
{
var endpoint = ValidateLiveEndpoint(credential, relativePath);
status?.Invoke(new(attempt == 0 ? CompanionPluginLiveState.Connecting : CompanionPluginLiveState.Reconnecting,
attempt == 0 ? "Connecting to Lumi…" : "Lumi is unavailable; reconnecting silently.", attempt));
using var socket = new ClientWebSocket();
socket.Options.SetRequestHeader("Authorization", $"LumiDevice {credential.DeviceId}.{credential.DeviceSecret}");
socket.Options.SetRequestHeader("User-Agent", "Lumi-Companion/1.0");
socket.Options.SetRequestHeader("X-Lumi-Companion-Plugin", _pluginId);
socket.Options.KeepAliveInterval = TimeSpan.FromSeconds(15);
await socket.ConnectAsync(endpoint, cancellationToken).ConfigureAwait(false);
attempt = 0;
log?.Invoke("plugin_live_connected", $"Plugin {_pluginId} connected to its authenticated Lumi live feed.");
status?.Invoke(new(CompanionPluginLiveState.Connected, "Live feed connected."));
await connected(new LiveChannel(socket), cancellationToken).ConfigureAwait(false);
if (socket.State == WebSocketState.Open)
await socket.CloseAsync(WebSocketCloseStatus.NormalClosure, "reconnect", cancellationToken).ConfigureAwait(false);
if (!cancellationToken.IsCancellationRequested) throw new WebSocketException("The Lumi live feed closed.");
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested) { break; }
catch (Exception error)
{
attempt += 1;
var detail = error.Message.Contains("403", StringComparison.Ordinal)
? "Lumi has not granted this paired device access to the plugin feed."
: error.Message.Contains("401", StringComparison.Ordinal)
? "Lumi no longer accepts this device pairing."
: "Lumi is unavailable; reconnecting silently.";
if (attempt == 1 || attempt % 10 == 0)
log?.Invoke("plugin_live_reconnecting", $"Plugin {_pluginId} live feed is reconnecting after {error.GetType().Name}.");
status?.Invoke(new(CompanionPluginLiveState.Reconnecting, detail, attempt));
var delay = TimeSpan.FromSeconds(Math.Min(30, Math.Pow(2, Math.Min(attempt, 5))));
await Task.Delay(delay, cancellationToken).ConfigureAwait(false);
}
}
status?.Invoke(new(CompanionPluginLiveState.Stopped, "Live feed stopped."));
}
private Uri ValidateLiveEndpoint(DeviceCredential credential, string relativePath)
{
var host = new Uri(credential.Host);
if (!CompanionTransportPolicy.IsPairedHostAllowed(host))
throw new InvalidDataException("Companion transport requires HTTPS unless this device was paired with localhost.");
var expectedPrefix = $"/plugins/{_pluginId}/";
if (!relativePath.StartsWith(expectedPrefix, StringComparison.Ordinal) ||
!Uri.TryCreate(relativePath, UriKind.Relative, out var relative))
throw new InvalidDataException($"Plugin '{_pluginId}' attempted to use an endpoint outside its Lumi plugin scope.");
var httpEndpoint = new Uri(host, relative);
if (!CompanionTransportPolicy.IsEndpointAllowed(httpEndpoint, host))
throw new InvalidDataException("The plugin endpoint does not match the paired Lumi host.");
var builder = new UriBuilder(httpEndpoint)
{
Scheme = httpEndpoint.Scheme == Uri.UriSchemeHttps ? "wss" : "ws",
Port = httpEndpoint.IsDefaultPort ? -1 : httpEndpoint.Port
};
return builder.Uri;
}
private sealed class LiveChannel(ClientWebSocket socket) : ICompanionPluginLiveChannel
{
private readonly SemaphoreSlim _sendLock = new(1, 1);
public async Task SendJsonAsync(object payload, CancellationToken cancellationToken = default)
{
var bytes = JsonSerializer.SerializeToUtf8Bytes(payload, ProtocolV1.JsonOptions);
if (bytes.Length > 64 * 1024) throw new InvalidDataException("Plugin live message exceeds the 64 KiB limit.");
await _sendLock.WaitAsync(cancellationToken).ConfigureAwait(false);
try { await socket.SendAsync(bytes, WebSocketMessageType.Text, true, cancellationToken).ConfigureAwait(false); }
finally { _sendLock.Release(); }
}
public async IAsyncEnumerable<string> ReadAllAsync([EnumeratorCancellation] CancellationToken cancellationToken = default)
{
var buffer = new byte[16 * 1024];
using var message = new MemoryStream();
while (!cancellationToken.IsCancellationRequested && socket.State == WebSocketState.Open)
{
var result = await socket.ReceiveAsync(buffer, cancellationToken).ConfigureAwait(false);
if (result.MessageType == WebSocketMessageType.Close) yield break;
if (result.MessageType != WebSocketMessageType.Text) continue;
if (message.Length + result.Count > 64 * 1024) throw new InvalidDataException("Lumi sent an oversized plugin live message.");
message.Write(buffer, 0, result.Count);
if (!result.EndOfMessage) continue;
yield return Encoding.UTF8.GetString(message.GetBuffer(), 0, (int)message.Length);
message.SetLength(0);
}
}
}
private static string ValidatePluginId(string value)
{
var clean = (value ?? string.Empty).Trim();

View File

@ -0,0 +1,46 @@
namespace Lumi.Companion.Core;
public sealed class SingleFlightOperation
{
private readonly object _gate = new();
private Task? _active;
private long _generation;
public bool IsRunning
{
get { lock (_gate) return _active is { IsCompleted: false }; }
}
public Task RunAsync(Func<CancellationToken, Task> operation, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(operation);
lock (_gate)
{
if (_active is { IsCompleted: false }) return _active;
var generation = ++_generation;
var completion = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
var task = completion.Task;
_active = task;
_ = RunCoreAsync(generation, operation, cancellationToken, completion);
return task;
}
}
private async Task RunCoreAsync(long generation, Func<CancellationToken, Task> operation, CancellationToken cancellationToken, TaskCompletionSource completion)
{
try
{
await operation(cancellationToken);
completion.TrySetResult();
}
catch (OperationCanceledException error) { completion.TrySetCanceled(error.CancellationToken); }
catch (Exception error) { completion.TrySetException(error); }
finally
{
lock (_gate)
{
if (_generation == generation) _active = null;
}
}
}
}

View File

@ -0,0 +1,47 @@
using System.Security.Cryptography;
using System.Text.Json;
using Lumi.Companion.Protocol;
namespace Lumi.Companion.Core;
public sealed record StreamTestRecoveryState(
string SessionId,
string ServiceType,
string ServiceSettingsJson,
DateTimeOffset CreatedAt);
public sealed class StreamTestRecoveryStore(string root)
{
private readonly string _path = Path.Combine(root, "stream-test.recovery");
private static readonly byte[] Entropy = "Lumi.Companion.StreamTestRecovery.v1"u8.ToArray();
public void Save(StreamTestRecoveryState state)
{
Directory.CreateDirectory(Path.GetDirectoryName(_path)!);
var clear = JsonSerializer.SerializeToUtf8Bytes(state, ProtocolV1.JsonOptions);
var protectedBytes = ProtectedData.Protect(clear, Entropy, DataProtectionScope.CurrentUser);
var temporary = $"{_path}.{Environment.ProcessId}.tmp";
try
{
File.WriteAllBytes(temporary, protectedBytes);
File.Move(temporary, _path, true);
}
finally
{
File.Delete(temporary);
CryptographicOperations.ZeroMemory(clear);
}
}
public StreamTestRecoveryState? Load()
{
if (!File.Exists(_path)) return null;
var protectedBytes = File.ReadAllBytes(_path);
var clear = ProtectedData.Unprotect(protectedBytes, Entropy, DataProtectionScope.CurrentUser);
try { return JsonSerializer.Deserialize<StreamTestRecoveryState>(clear, ProtocolV1.JsonOptions); }
finally { CryptographicOperations.ZeroMemory(clear); }
}
public bool Exists => File.Exists(_path);
public void Remove() => File.Delete(_path);
}

View File

@ -0,0 +1,14 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0-windows10.0.19041.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<RollForward>Major</RollForward>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="../../src/Lumi.Companion.Core/Lumi.Companion.Core.csproj" />
<ProjectReference Include="../../plugins/Lumi.Companion.Overlay/Lumi.Companion.Overlay.csproj" />
<ProjectReference Include="../../plugins/Lumi.Companion.SongOverlay/Lumi.Companion.SongOverlay.csproj" />
</ItemGroup>
</Project>

View File

@ -0,0 +1,162 @@
using Lumi.Companion.Abstractions;
using Lumi.Companion.Core;
using Lumi.Companion.Overlay;
using Lumi.Companion.SongOverlay;
static void Assert(bool condition, string message)
{
if (!condition) throw new InvalidOperationException(message);
}
var operation = new SingleFlightOperation();
var calls = 0;
await operation.RunAsync(_ => { calls += 1; return Task.CompletedTask; });
Assert(calls == 1 && !operation.IsRunning, "A successful update-style operation did not reset.");
await operation.RunAsync(_ => { calls += 1; return Task.CompletedTask; });
Assert(calls == 2 && !operation.IsRunning, "A no-update-style repeat did not execute.");
try
{
await operation.RunAsync(_ => throw new InvalidOperationException("expected"));
}
catch (InvalidOperationException error) when (error.Message == "expected") { }
await operation.RunAsync(_ => { calls += 1; return Task.CompletedTask; });
Assert(calls == 3 && !operation.IsRunning, "A failed operation left stale single-flight state.");
using (var cancelled = new CancellationTokenSource())
{
cancelled.Cancel();
try { await operation.RunAsync(token => Task.Delay(1, token), cancelled.Token); }
catch (OperationCanceledException) { }
}
await operation.RunAsync(_ => { calls += 1; return Task.CompletedTask; });
Assert(calls == 4 && !operation.IsRunning, "A cancelled operation left stale single-flight state.");
var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
var concurrentCalls = 0;
var first = operation.RunAsync(async _ => { concurrentCalls += 1; await release.Task; });
var second = operation.RunAsync(async _ => { concurrentCalls += 1; await Task.Yield(); });
Assert(operation.IsRunning && concurrentCalls == 1, "Rapid update checks started more than one operation.");
release.SetResult();
await Task.WhenAll(first, second);
Assert(concurrentCalls == 1 && !operation.IsRunning, "Rapid update checks did not share and reset the active operation.");
var sanitized = CompanionLogSanitizer.Sanitize(
"LumiDevice device-id.private-device-secret Authorization=private-value https://lumi.test/?token=private-token Cookie: session=private-cookie"
);
Assert(!sanitized.Contains("private-value"), "Companion logs retained an authorization value.");
Assert(!sanitized.Contains("private-cookie"), "Companion logs retained a cookie.");
Assert(!sanitized.Contains("private-token"), "Companion logs retained a query token.");
Assert(!sanitized.Contains("private-device-secret"), "Companion logs retained a paired-device credential.");
Assert(CompanionLogSanitizer.NormalizeEvent("Update Failed!") == "update_failed", "Companion event normalization drifted.");
Assert(CompanionLogSanitizer.LevelForEvent("update_failed") == "error", "Companion failed events must use the error level.");
var anchored = new OverlayContainerSettings
{
Anchor = OverlayAnchor.Top,
PaddingTop = 20,
PaddingLeft = 40,
PaddingRight = 60,
PaddingBottom = 999,
Width = 5000,
Height = 300
};
var rect = AnchorLayout.Calculate(anchored, 1920, 1080);
Assert(anchored.PaddingBottom == 0, "Top-side anchors must zero disabled bottom padding.");
Assert(rect.Width == 1820 && rect.X == 40 && rect.Y == 20, "Physical-pixel layout did not clamp and center symmetrically inside valid padding.");
var now = DateTimeOffset.UnixEpoch;
var queueSettings = new OverlayContainerSettings { QueueLimit = 2, EntryDurationMs = 100, LifetimeMs = 200, ExitDurationMs = 100 };
var queue = new OverlayLifecycleQueue(() => queueSettings, () => now);
queue.Add("one", "chat", "one");
queue.Add("two", "chat", "two");
queue.Add("three", "chat", "three");
Assert(queue.Items[0].State == OverlayItemState.Exiting && queue.Items.Count == 3,
"Overflow must animate the oldest item out while keeping it rendered.");
now += TimeSpan.FromMilliseconds(100);
queue.Advance();
Assert(queue.Items.Count == 2 && queue.Items.All(item => item.State == OverlayItemState.Visible),
"Entry and exit lifecycle transitions did not advance deterministically.");
now += TimeSpan.FromMilliseconds(201);
queue.Advance();
Assert(queue.Items.All(item => item.State == OverlayItemState.Exiting), "Lifetime must begin only after entry completes.");
for (var index = 0; index < 20; index++) queue.Add($"burst-{index}", "chat", index);
Assert(queue.Items.Count <= queueSettings.QueueLimit * 2, "Burst handling must keep the rendered lifecycle queue strictly bounded.");
var settingsPath = Path.Combine(Path.GetTempPath(), $"lumi-overlay-test-{Guid.NewGuid():N}", "settings.json");
var overlayStore = new OverlaySettingsStore(settingsPath);
await overlayStore.LoadAsync();
overlayStore.Current.Sources.Add(new OverlaySourceSetting
{
Id = "twitch:renamed-channel",
Platform = "twitch",
Label = "Previously configured channel",
Selected = true
});
overlayStore.Current.Unified.Anchor = OverlayAnchor.BottomRight;
overlayStore.Current.Unified.PaddingTop = 42;
await overlayStore.SaveAsync(overlayStore.Current);
var restoredStore = new OverlaySettingsStore(settingsPath);
await restoredStore.LoadAsync();
Assert(restoredStore.Current.Sources.Single().Selected && restoredStore.Current.Sources.Single().Label == "Previously configured channel",
"Unavailable source selections were not retained by atomic local settings persistence.");
Assert(restoredStore.Current.Unified.PaddingTop == 0,
"Settings normalization must clear padding that is invalid for the selected anchor.");
var unsavedDraft = restoredStore.Current.CreateCopy();
unsavedDraft.ChatPercentage = 60;
unsavedDraft.Sources[0].Selected = false;
Assert(restoredStore.Current.ChatPercentage != unsavedDraft.ChatPercentage && restoredStore.Current.Sources[0].Selected,
"Editable overlay drafts must not mutate or get replaced by the last saved settings.");
Assert(!CompanionPluginNavigation.UsesNestedNavigation(
[new CompanionPluginPage("SongOverlay", "Overview & settings")]) &&
CompanionPluginNavigation.UsesNestedNavigation(
[new CompanionPluginPage("Transcription", "Capture"), new CompanionPluginPage("Test", "Test")]),
"Single-page plugins must be direct navigation entries while multi-page plugins remain grouped.");
var defaultMediaSources = MediaSourceCatalog.Defaults();
Assert(defaultMediaSources.SetEquals(["wmp", "youtube", "spotify"]),
"Song discovery defaults must enable only Windows Media Player, YouTube, and Spotify.");
Assert(MediaSourceCatalog.Detect("Spotify.exe", "Track", "Artist", "") == "spotify" &&
MediaSourceCatalog.Detect("chrome.exe", "Track · SoundCloud", "Artist", "") == "soundcloud" &&
MediaSourceCatalog.Detect("chrome.exe", "Unlabelled browser track", "Artist", "") == "youtube" &&
MediaSourceCatalog.Detect("vlc.exe", "Track", "", "") == "vlc",
"Windows media-session discovery did not classify desktop and browser sources independently.");
Assert(MediaSourceCatalog.DetectEnabled("chrome.exe", "Unlabelled track", "Artist", "",
new HashSet<string>(["soundcloud"], StringComparer.Ordinal)) == "soundcloud",
"An independently enabled browser source must remain discoverable when Windows withholds the tab origin.");
Assert(new CaptureExclusionStatus(CaptureExclusionState.Unsupported, "unsupported").State == CaptureExclusionState.Unsupported,
"Capture-exclusion unsupported state must remain distinct from failure.");
var visibilitySettings = new LumiOverlaySettings
{
Visibility = OverlayVisibilityMode.WhenLive,
Sources = [new() { Id = "twitch:lumi", Platform = "twitch", Label = "Lumi", Selected = true }]
};
var liveSources = new[] { new OverlaySource("twitch:lumi", "twitch", "Lumi", true, true, "live") };
Assert(OverlayVisibilityPolicy.Automatic(visibilitySettings, liveSources), "A selected live Twitch source must activate When live.");
Assert(!OverlayVisibilityPolicy.Automatic(visibilitySettings,
[new OverlaySource("discord:guild:channel", "discord", "Discord", true, true, "connected")]),
"Discord must never activate When live by itself.");
Assert(OverlayVisibilityPolicy.Effective(visibilitySettings, [], true, null), "Configuration preview must override live-state visibility.");
visibilitySettings.Enabled = false;
Assert(OverlayVisibilityPolicy.Effective(visibilitySettings, [], true, null), "Configuration preview must remain available while the runtime is disabled.");
visibilitySettings.Enabled = true;
Assert(!OverlayVisibilityPolicy.Effective(visibilitySettings, liveSources, false, false), "A manual hidden override must win until automatic visibility is restored.");
var automaticPresentation = OverlayVisibilityPresenter.Describe(visibilitySettings, true, false, null);
Assert(automaticPresentation.Visible && automaticPresentation.Action == "Hide temporarily" &&
automaticPresentation.State.Contains("automatic", StringComparison.Ordinal),
"Visibility presentation must separate the current automatic state from the button action.");
var overridePresentation = OverlayVisibilityPresenter.Describe(visibilitySettings, true, false, false);
Assert(!overridePresentation.Visible && overridePresentation.Action == "Restore automatic visibility",
"Visibility presentation must identify a temporary override and explain how to leave it.");
var previewPresentation = OverlayVisibilityPresenter.Describe(visibilitySettings, false, true, null);
Assert(previewPresentation.Visible && !previewPresentation.CanToggle && previewPresentation.Action.Contains("End preview", StringComparison.Ordinal),
"Preview-controlled visibility must be explicit and must not offer an ineffective override action.");
var monitorFallback = OverlayMonitorPolicy.Resolve("missing", ["primary", "second"], "primary");
Assert(monitorFallback.MonitorId == "primary" && monitorFallback.UsedFallback && monitorFallback.Warning is not null,
"Monitor loss must fall back without erasing the saved monitor or hiding the warning.");
Assert(CompanionLiveReconnectPolicy.Delay(1) == TimeSpan.FromSeconds(2) &&
CompanionLiveReconnectPolicy.Delay(99) == TimeSpan.FromSeconds(30),
"Live feed reconnect backoff must be bounded and deterministic.");
Directory.Delete(Path.GetDirectoryName(settingsPath)!, true);
Console.WriteLine("Companion core, native overlay layout/lifecycle, settings persistence, and capture-status regression checks passed.");

112
docs/LOGGING_STANDARD.md Normal file
View File

@ -0,0 +1,112 @@
# Logging standard
Lumi uses the structured logger in `src/services/logger.js` for operational
events. It stores searchable entries for the admin UI, attaches request context,
redacts common credential fields, and publishes live admin updates.
## Operational logging
Core services create a named logger:
```js
const { createLogger } = require("./logger");
const logger = createLogger("core:updater", { category: "updates" });
logger.info("Update check completed", { version }, {
event: "update_check_completed"
});
```
Plugins receive a plugin-scoped `logger` in their `init()` dependencies. Use
that injected logger when practical. A plugin module that must log outside
`init()` may create a logger named `plugin:<plugin-id>`.
Use these source prefixes consistently:
- `core:<component>` for Lumi core and WebUI services.
- `platform:<provider>` for Discord, Twitch, YouTube, and similar integrations.
- `plugin:<plugin-id>` for bundled and locally installed plugins.
- `companion:<component>` for the desktop Companion host.
Give each logger a stable default category such as `lifecycle`, `http`,
`integration`, `command`, `automation`, `security`, `updates`, or `plugin`.
Every operational call must provide a stable, lowercase `snake_case` event ID.
Messages are for people and may improve over time; event IDs are for filtering
and automation and should remain stable.
Use:
- `debug` for detailed, low-value troubleshooting information.
- `info` for meaningful lifecycle and administrative actions.
- `warn` for degraded behavior that Lumi can continue through.
- `error` for a failed operation that may require attention.
Pass an `Error` as the details argument when one is available. The logger
preserves its stack trace while applying credential redaction:
```js
logger.error("Plugin refresh failed", error, {
event: "plugin_refresh_failed"
});
```
The metadata argument only accepts `source`, `category`, `event`, and
`requestId`. Operational fields such as `plugin_id`, `user_id`, `status`, or
`duration_ms` belong in the details argument:
```js
logger.warn("Plugin health check degraded", {
plugin_id: plugin.id,
status: health.status
}, {
event: "plugin_health_degraded"
});
```
Do not include passwords, tokens, cookies, pairing secrets, authorization
headers, signature fragments, full request bodies, or full third-party payloads
in messages or details. Record a bounded summary with IDs, status, counts, and
safe error text instead. Redaction is a safety net, not a reason to collect
secrets.
## Metrics and high-frequency events
Do not write high-frequency counters or per-frame events to the operational log.
There is no global metrics API.
A feature may retain its own bounded metrics only when it also owns the storage,
retention, and inspection UI. Lumi AI is one example:
`plugins/lumi_ai/backend/metrics.js` aggregates AI timings and retains bounded
work history. Operational failures in such a feature still belong in the core
logger.
Feature-owned diagnostic logs follow the same credential and payload rules.
They must have explicit size/age retention, sanitize recursively before writing,
and remain admin-only. Transcription worker diagnostics are kept separately
because they are high-volume troubleshooting data; they are not a substitute
for operational warnings and errors in the core logger.
The desktop Companion cannot write directly to the server database. Its local
JSON Lines logs therefore carry the same `level`, `source`, `category`, `event`,
and human-readable `message` shape, use the shared
`CompanionLogSanitizer`, and enforce local retention.
The native OBS Bridge must use OBS's `blog()` facility so its messages remain in
the operator's OBS log. Prefix each message with `[Lumi Companion]` and a stable
`event=<snake_case>` field, and never include IPC payloads or credentials.
## Console output
Direct `console.*` calls are captured after `hookConsole()` starts, but they lose
the useful source and event metadata of a named logger. Prefer a named logger in
runtime code, including startup and shutdown paths. Console output remains
reasonable in standalone verification and build scripts where the terminal is
the intended consumer. Browser-side console diagnostics and isolated worker
sandboxes are outside the server operational-log boundary.
## Retention and context
Operational logs default to 30 days and at most 100,000 entries. Request
correlation is inherited through `withLogContext()` and logger `run()` scopes.
The logger redacts common sensitive keys and credential-looking text before an
entry is stored or published.

View File

@ -0,0 +1,54 @@
# Companion native overlay
Lumi Companion includes a Windows-native monitor overlay under **Plugins → Lumi
Overlay**. It is independent of Lumi's OBS browser-source overlays: it uses
Avalonia controls in a transparent, topmost, click-through window and never
loads HTML or provider credentials.
## Data and authorization
The server endpoint is `/plugins/lumi_overlay/live`. It authenticates the same
paired device credential as Companion and requires `overlay.read.v1`. New
pairings receive the capability. Existing active devices receive it through a
one-time migration; a later capability removal is not silently undone.
Lumi reuses its normalized Twitch, YouTube, and Discord chat/event streams.
Selections are filtered on the server, messages are sanitized and bounded, and
a connection receives live items only—no history is replayed after startup or
reconnect. Missing provider scopes disable only their affected event types and
are reported in feed status.
The native renderer uses the same normalized badge, emote, and Discord media
payloads as Lumi's OBS browser overlay. HTTPS artwork is downloaded with strict
size, dimension, frame-count, and decoded-pixel limits. Animated GIF/WebP
emotes and images retain their bounded frame timing; unsupported video-only
embeds use their supplied preview image when available.
## Window behavior
The selected monitor, layout, styling, sources, event types, visibility mode,
animation lifecycle, and hotkeys are stored atomically under the current
Windows user's Companion data directory. A missing monitor temporarily falls
back to the primary screen without changing the saved monitor.
The overlay reapplies click-through, non-activating, tool-window and topmost
styles after display, power, and Windows session changes. It calls
`SetWindowDisplayAffinity(WDA_EXCLUDEFROMCAPTURE)` and reports the result in the
settings page. This is best-effort Windows protection: OBS and other capture
software can use capture paths that do not honor display affinity.
`When live` is activated only by a selected Twitch or YouTube source. Discord
does not activate it. Sample preview and the visibility override temporarily
take precedence, then restore the saved mode.
## Verification
From PowerShell:
```powershell
companion/scripts/verify-lumi-overlay.ps1
```
The regular `npm run verify:all` suite also exercises the server protocol,
filtering, no-backlog behavior, queue bounds, capability migration, and
revocation behavior.

View File

@ -32,10 +32,21 @@ portable fallback.
7. Use the dedicated transcription test when measuring real speech accuracy,
confidence, latency, and input level.
The Companion Transcription page and tray menu expose a persistent **Generate
and include captions** switch. When it is off, Companion does not queue or send
microphone audio for recognition, does not start a live transcription session,
and removes caption inclusion from a private stream test. The video-only private
test remains available. Path and benchmark tests remain disabled until captions
are turned back on.
Production device HTTP and WebSocket traffic requires HTTPS/WSS. Plain
HTTP/WebSocket is allowed only for a device paired from the exact matching
loopback Lumi origin and only while both sides remain on loopback.
Lumi recognizes HTTPS terminated by a reverse proxy on loopback or an isolated
private proxy network. Forwarded protocol headers received directly from public
addresses are ignored, so only a trusted proxy can mark a request as HTTPS.
## Models and workers
The Admin page installs checksum-pinned model and worker artifacts only after
@ -54,7 +65,9 @@ workers are built with `plugins/lumi_transcription/scripts/build-worker.ps1`.
- Live sessions require an active OBS stream. Tests use simulated delivery and
never publish captions to Twitch.
- Disconnects pause delivery and retain the session/model for a bounded grace
period so a short reconnect can resume safely.
period so a short reconnect can resume safely. Companion retries a paired Lumi
host silently with bounded backoff and resumes its authenticated runtime state
after reconnecting.
- Manual benchmark completion waits for the final confidence-bearing caption;
silence ends a test after ten seconds.
- Device revocation takes effect on the next authenticated request or
@ -66,6 +79,20 @@ The Admin summary reports Companion, device, inference, and session health.
Detailed benchmark records remain inspectable for one hour and include
word/phrase latency and confidence views.
## OBS Bridge and Stream Testing
The same paired Companion identity and same-user OBS Bridge are also used for
private Stream Testing. Lumi issues the expiring RTMP/RTMPS destination from the
validated pairing hostname; plugins do not re-authenticate and Companion does
not invent or accept another host. Before changing OBS, Companion stores the
complete service snapshot in its DPAPI-protected recovery record. Every terminal
path uses that record until OBS confirms the exact service has been restored.
MediaMTX is installed, verified, configured, and supervised by Lumi Core on the
server. It is not shipped in Companion and does not run inference or transcode
media. See [`stream-testing.md`](stream-testing.md) for transport, firewall,
managed-runtime, playback, and recovery details.
## Verification
```powershell

View File

@ -5,6 +5,19 @@ normal Lumi session, role checks, layout, forms, lists, modals, and confirmation
flow. Browser Source pages use a separate transparent document with no Lumi
navigation or authenticated website data.
## Media source behavior
Video sources render only their pixels. Browser controls, poster chrome, and
fallback backgrounds are never exposed in the public overlay. A video remains
hidden until the media element reports real playback and is hidden again after
end, stop, clear, or error. Playback controls stay in the Lumi inspector.
Audio sources are managed outputs rather than visual canvas objects. Their
play, pause, stop, seek, volume, mute, loop, test, and removal controls live in
the source list/inspector. Public overlays contain no visible audio UI, and old
audio position, size, layer, crop, alignment, opacity, and anchor fields are
ignored for compatibility.
## Editing an overlay
The editor uses the same basic model as OBS: scenes contain ordered sources, and

View File

@ -11,6 +11,10 @@ The frontend must not declare placeholder permissions, allowed plugins, or
sensitivity. Editable fields reference a trusted `field_id`, and the server
uses that field policy to decide which placeholders are available.
Platform membership and role duration placeholders, including their embedded
`{format.YY-MM-DD}` grammar, are documented in
[platform-tenure.md](platform-tenure.md).
Core and plugins can register:
- placeholder definitions with metadata and a resolver function

44
docs/platform-tenure.md Normal file
View File

@ -0,0 +1,44 @@
# Platform tenure
Lumi records platform membership and role periods as immutable UTC intervals. The same history powers custom-command placeholders, the **Platform tenure** cards on `/stats`, the **Platform tenure** section on `/leaderboards`, and the admin diagnostics page at `/admin/platform-tenure`.
## Supported statistics
| Platform | Statistics | Start timestamp |
| --- | --- | --- |
| Twitch | Follow, subscription, moderator, editor, VIP | Platform timestamp where Twitch supplies one; otherwise first reliable observation |
| YouTube | Paid channel membership and live-chat moderator | Membership event time where available; otherwise first reliable live-chat observation |
| Discord | Server membership and server boost | Discord `joinedTimestamp` and `premiumSinceTimestamp` where available |
YouTube does not expose a reliable timestamp for a viewer's ordinary public channel subscription through the existing live-chat integration. `youtube.user.subscriber_age` therefore represents paid channel membership (sponsor/member state), not a fabricated public-subscription age. YouTube roles are observed only when the live-chat API reports them.
Temporary API failures do not close intervals. Lumi closes missing states only after a complete successful platform snapshot or a reliable end event. Unlinking a platform identity does not delete interval history.
## Placeholders
The current uninterrupted interval and cumulative recorded history are exposed as `current` and `total`:
```text
{{twitch.user.follow_age.current}}
{{twitch.user.subscriber_age.total}}
{{twitch.user.mod_age.current}}
{{twitch.user.editor_age.total}}
{{twitch.user.vip_age.current}}
{{youtube.user.subscriber_age.current}}
{{youtube.user.mod_age.total}}
{{discord.user.member_age.current}}
{{discord.user.nitro_age.total}}
```
The current command context selects the platform user and channel or guild scope. Lumi never combines values across Twitch channels, YouTube channels, or Discord servers.
Append an embedded format to select duration units:
```text
{{twitch.user.follow_age.total}.{format.YY-MM-DD-HH-mm-ss}}
{{discord.user.member_age.current}.{format.DD-HH-mm}}
```
Units are case-sensitive: `Y`, `YY`, or `YYYY` means years; `M` or `MM` months; `D` or `DD` days; `H` or `HH` hours; `m` or `mm` minutes; and `s` or `ss` seconds. Repetition identifies the unit and does not add zero-padding. Units must be ordered largest to smallest and may appear once.
Lumi uses deterministic cumulative-duration units: one year is 365 days and one month is 30 days. Omitted time flows into the next smaller selected unit, so a format containing only `mm` returns total complete minutes. Time below the smallest selected unit is truncated. Zero units are omitted; an all-zero result is empty.

146
docs/stream-testing.md Normal file
View File

@ -0,0 +1,146 @@
# Private stream testing
Lumi Stream Testing previews the real OBS output without publishing it to the
normal stream destination. Start it from the core **Stream testing** page in
Lumi Companion, then watch it at **Admin > Stream testing**.
## Architecture
- The existing paired-device WebSocket authenticates session creation, OBS
metrics, caption cues, stop requests, and server-ended notifications.
- The existing same-user OBS Bridge snapshots the complete current OBS service,
temporarily installs an expiring custom service, reports real output facts,
and restores the snapshot.
- Companion protects the minimum recovery record with Windows DPAPI and writes
it atomically before OBS is redirected. Normal stop, server failure, expiry,
Companion exit, WebSocket loss, and OBS restart all use that restore path.
- Core lazily starts a Lumi-managed MediaMTX process. MediaMTX accepts the
private RTMP(S) publisher and repackages the original source into low-latency
HLS without transcoding, scaling, a GPU requirement, or an adaptive ladder.
- The MediaMTX API, metrics, and HLS listeners bind only to loopback. The Admin
WebUI accesses HLS through an authenticated, no-store, same-origin streaming
proxy. RTSP, WebRTC, SRT, playback, pprof, and recording are disabled.
- Companion explicitly starts a server-hosted `stream_test` transcription
session before redirecting OBS. It reuses the selected source and model while
keeping normal live-caption delivery disabled. Interim revisions progressively
reveal each caption in the private player; a finalized caption remains for a
length-aware readability window, then ends when stale or when the next
utterance replaces it. The player's Captions control toggles a wrapping,
multiline Lumi caption layer immediately as revisions arrive instead of
delaying display against the HLS media timeline. No second transcription
pipeline and no local speech inference are added.
Media segments stay in bounded MediaMTX memory and roll out of the short live
window. Lumi does not retain a raw stream recording or create temporary HLS
files. Session state, captions, and diagnostic facts disappear when the test
ends or Lumi restarts.
## Managed setup
No operating-system package installation, FFmpeg installation, or executable
path is required. Open **Admin > Stream testing** and choose **Install
MediaMTX**. Lumi then:
1. downloads the pinned official Windows x64 archive only after confirmation;
2. enforces download and extraction size limits;
3. verifies the exact SHA-256 checksum and archive layout;
4. probes the executable and exact expected version; and
5. atomically installs it while retaining the previous installation until the
replacement is known to be valid.
**Repair installation** repeats the same verified workflow. **Run health
check** starts MediaMTX with deny-by-default paths and tests its loopback API,
metrics, and HLS listeners; opening the page alone never downloads or starts a
runtime. `LUMI_MEDIAMTX_PATH` remains an advanced manual override for another
platform or operator-managed binary, but normal Windows installations do not
need it.
The managed release and checksum source are recorded in
`src/services/mediamtx-runtime-manifest.json`. MediaMTX's licence is retained in
the downloaded runtime and documented in
[`third-party/MediaMTX-MIT.txt`](third-party/MediaMTX-MIT.txt).
## Ingest network configuration
Lumi derives the network policy from the authenticated Companion pairing
origin. A Companion paired through `localhost`, `127.0.0.1`, or `::1` always
receives direct loopback RTMP on the configured local listener port. A
non-local Companion must have paired through HTTPS and always receives RTMPS
at the same paired Lumi hostname.
In production, Nginx Proxy Manager owns the public certificate and terminates
TLS:
```text
OBS → rtmps://<paired Lumi hostname>:1936
→ Nginx Proxy Manager decrypts TLS
→ plain RTMP to Lumi:19350
```
MediaMTX binds plain RTMP on `0.0.0.0:19350` for a production session. It never
requests, loads, renews, or stores a TLS certificate. Configure an NPM TCP
stream that listens on public port `1936` with TLS enabled and forwards to the
Lumi host on port `19350`. The WebUI reverse-proxy settings can change both
ports; values are stored in Lumi's settings database and apply to the next
session.
The collapsed **Reverse-proxy ingest** section in **Admin > Stream testing**
also provides **Check external access**. The check connects to the current
public Lumi hostname and port with TLS and SNI, validates the public
certificate, then completes an RTMP handshake through NPM to MediaMTX. A
server-side failure can still mean that the router does not support NAT
loopback/hairpinning, so confirm the path from an external network when the NPM
and firewall configuration otherwise look correct.
Every session receives an exact `lumi-test/<uuid>` path and high-entropy
publisher credentials. Only that path can be published, and the generated
MediaMTX configuration contains SHA-256 credential hashes rather than
plaintext credentials.
RTMPS is required for every non-local pairing. Unencrypted RTMP is advertised
only to an authenticated loopback pairing. Use the existing paired Lumi
hostname; no extra Stream Testing domain is required. Restrict direct access
to the local MediaMTX listener with the host firewall so production publishers
must enter through NPM.
Optional session limits remain:
```text
LUMI_STREAM_TEST_MAX_MS=1800000
LUMI_STREAM_TEST_INACTIVITY_MS=45000
```
## Safety and recovery
Companion refuses to start while OBS is streaming or recording. During a test
its navigation status reads **PRIVATE TEST ACTIVE**. The DPAPI recovery file is
removed only after OBS confirms that the saved service type and complete
settings JSON were restored and saved.
If OBS was closed or unavailable during cleanup, open OBS and leave Companion
running. Automatic restoration resumes when the same-user bridge reconnects.
The **Repair restoration** action retries the same operation and does not create
a new session.
The receiver stops on maximum duration, lack of inbound media, process failure,
server shutdown, Companion disconnect, or administrator stop. OBS status
messages do not reset the media-inactivity clock. Only one session is allowed.
MediaMTX is reconfigured to a deny-by-default path set after every session.
## Diagnostics and verification
The Admin page reports real OBS bitrate, dropped frames, congestion, MediaMTX
path readiness, media tracks, inbound/outbound bytes, HLS viewers, discarded
frames, player latency, buffer health, stalls, errors, and caption delay.
Only an allowlist of MediaMTX API and metrics fields is exposed. Publisher
credentials are redacted from status, timeline entries, process logs, and
errors.
The runtime health check replaces the former transcoded synthetic test pattern.
Focused verification uses a fake MediaMTX process with real loopback API,
metrics, and HLS listeners:
```text
npm run verify:stream-testing
dotnet run --project companion/tests/Lumi.Companion.Core.Tests/Lumi.Companion.Core.Tests.csproj
```

24
docs/third-party/MediaMTX-MIT.txt vendored Normal file
View File

@ -0,0 +1,24 @@
MediaMTX v1.18.2
Project: https://github.com/bluenviron/mediamtx
MIT License
Copyright (c) 2019 aler9
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@ -14,7 +14,7 @@ editable: false
Lumi is the core web UI and bot runtime.
## Runtime
Package: lumi-bot
Version: 0.2.27
Version: 0.3.11
## Routes
- POST /api/diagnostics/v1/run
- GET /api/events
@ -93,6 +93,19 @@ Version: 0.2.27
- POST /admin/theming/custom/:id/delete
- POST /admin/theming
- GET /admin/diagnostics
- GET /admin/platform-tenure
- GET /admin/stream-testing
- POST /admin/stream-testing/reverse-proxy
- POST /admin/stream-testing/reverse-proxy/check
- GET /admin/stream-testing/status
- POST /admin/stream-testing/stop
- POST /admin/stream-testing/runtime/install
- POST /admin/stream-testing/runtime/repair
- POST /admin/stream-testing/runtime/health
- POST /admin/stream-testing/player-metrics
- GET /admin/stream-testing/media/:id/captions.vtt
- GET /admin/stream-testing/media/:id/*
- GET /admin/stream-testing/hls.js
- POST /admin/diagnostics/run
- POST /admin/diagnostics/access/renew
- POST /admin/diagnostics/access/revoke
@ -908,6 +921,123 @@ Version: 0.2.27
- Side effects: Usually read-only.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations.
### GET /admin/platform-tenure
- Purpose: Renders the admin platform tenure WebUI page.
- Inputs: query: `platform`, `scope`, `type`, `user`
- Response format: HTML page rendered from an EJS view
- Access: admin access expected
- Side effects: Usually read-only.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations.
### GET /admin/stream-testing
- Purpose: Renders the admin stream testing WebUI page.
- Inputs: No request parameters detected by static analysis.
- Response format: HTML page rendered from an EJS view
- Access: admin access expected
- Side effects: Usually read-only.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations.
### POST /admin/stream-testing/reverse-proxy
- Purpose: Processes the admin stream testing reverse proxy action and stores or applies submitted form data.
- Inputs: body: full submitted body is passed to a helper; exact fields are defined by the matching form/service
- Response format: HTTP redirect after handling the request
- Access: admin access expected
- Side effects: writes or mutates server-side state
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations. Errors are caught and usually returned as a flash message, JSON error, or error page. Most non-API POST routes are browser form submissions and usually redirect after completion.
### POST /admin/stream-testing/reverse-proxy/check
- Purpose: Provides admin stream testing reverse proxy check data as JSON.
- Inputs: No request parameters detected by static analysis.
- Response format: JSON response
- Access: admin access expected
- Side effects: Action route; side effects were not detected statically.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations. Errors are caught and usually returned as a flash message, JSON error, or error page. Most non-API POST routes are browser form submissions and usually redirect after completion.
### GET /admin/stream-testing/status
- Purpose: Provides admin stream testing status data as JSON.
- Inputs: No request parameters detected by static analysis.
- Response format: JSON response
- Access: admin access expected
- Side effects: Usually read-only.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations.
### POST /admin/stream-testing/stop
- Purpose: Provides admin stream testing stop data as JSON.
- Inputs: body: full submitted body is passed to a helper; exact fields are defined by the matching form/service
- Response format: JSON response
- Access: admin access expected
- Side effects: Action route; side effects were not detected statically.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations. Errors are caught and usually returned as a flash message, JSON error, or error page. Most non-API POST routes are browser form submissions and usually redirect after completion.
### POST /admin/stream-testing/runtime/install
- Purpose: Provides admin stream testing runtime install data as JSON.
- Inputs: body: full submitted body is passed to a helper; exact fields are defined by the matching form/service
- Response format: JSON response
- Access: admin access expected
- Side effects: writes or mutates server-side state
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations. Errors are caught and usually returned as a flash message, JSON error, or error page. Most non-API POST routes are browser form submissions and usually redirect after completion.
### POST /admin/stream-testing/runtime/repair
- Purpose: Provides admin stream testing runtime repair data as JSON.
- Inputs: body: full submitted body is passed to a helper; exact fields are defined by the matching form/service
- Response format: JSON response
- Access: admin access expected
- Side effects: Action route; side effects were not detected statically.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations. Errors are caught and usually returned as a flash message, JSON error, or error page. Most non-API POST routes are browser form submissions and usually redirect after completion.
### POST /admin/stream-testing/runtime/health
- Purpose: Returns runtime health information.
- Inputs: No request parameters detected by static analysis.
- Response format: JSON response
- Access: admin access expected
- Side effects: Action route; side effects were not detected statically.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations. Errors are caught and usually returned as a flash message, JSON error, or error page. Most non-API POST routes are browser form submissions and usually redirect after completion.
### POST /admin/stream-testing/player-metrics
- Purpose: Provides admin stream testing player metrics data as JSON.
- Inputs: body: full submitted body is passed to a helper; exact fields are defined by the matching form/service
- Response format: JSON response
- Access: admin access expected
- Side effects: Action route; side effects were not detected statically.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations. Errors are caught and usually returned as a flash message, JSON error, or error page. Most non-API POST routes are browser form submissions and usually redirect after completion.
### GET /admin/stream-testing/media/:id/captions.vtt
- Purpose: Handles admin stream testing media id captions vtt.
- Inputs: path params: `id`
- Response format: HTML or data response; exact format was not detected statically.
- Access: admin access expected
- Side effects: Usually read-only.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations.
### GET /admin/stream-testing/media/:id/*
- Purpose: Handles admin stream testing media id *.
- Inputs: path params: `id`
- Response format: HTML or data response; exact format was not detected statically.
- Access: admin access expected
- Side effects: Usually read-only.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations.
### GET /admin/stream-testing/hls.js
- Purpose: Handles admin stream testing hls js.
- Inputs: No request parameters detected by static analysis.
- Response format: static file response
- Access: admin access expected
- Side effects: Usually read-only.
- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations.
### POST /admin/diagnostics/run
- Purpose: Processes the admin diagnostics run action and stores or applies submitted form data.

View File

@ -14,7 +14,7 @@ editable: false
Auto-create managed voice channels from lobby rooms.
## Metadata
Plugin ID: auto-vc
Version: 0.1.6
Version: 0.1.7
Default state: enabled
## Web Routes
- /plugins/auto-vc

View File

@ -0,0 +1,26 @@
---
id: plugin.lumi_overlay
title: Lumi Overlay Feed
scope: plugins
status: active
priority: 10
visibility: user
category: Plugin
tags: plugin, lumi_overlay
generated: true
editable: false
---
# Lumi Overlay Feed
Authenticated live chat and event feed for the native Lumi Companion monitor overlay.
## Metadata
Plugin ID: lumi_overlay
Version: 0.1.1
Default state: enabled
## Web Routes
- No plugin routes detected.
## Route Reference
No route details detected.
## Commands
- No plugin command triggers detected.
## Source
Plugin folder: plugins/lumi_overlay

View File

@ -14,7 +14,7 @@ editable: false
Server-hosted whisper.cpp transcription for Lumi Companion and OBS closed captions.
## Metadata
Plugin ID: lumi_transcription
Version: 0.1.0
Version: 0.2.8
Default state: enabled
## Web Routes
- /plugins/lumi_transcription

View File

@ -14,7 +14,7 @@ editable: false
Provider-neutral song state, chat announcements, and a customizable Lumi Overlay source.
## Metadata
Plugin ID: now_playing
Version: 0.1.2
Version: 0.1.3
Default state: enabled
## Web Routes
- /plugins/now_playing

11
package-lock.json generated
View File

@ -1,12 +1,12 @@
{
"name": "lumi-bot",
"version": "0.2.27",
"version": "0.3.11",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "lumi-bot",
"version": "0.2.27",
"version": "0.3.11",
"dependencies": {
"adm-zip": "^0.6.0",
"better-sqlite3": "^11.5.0",
@ -15,6 +15,7 @@
"ejs": "^3.1.10",
"express": "^4.19.2",
"express-session": "^1.18.1",
"hls.js": "^1.6.16",
"multer": "^2.2.0",
"tmi.js": "^1.8.5",
"ws": "^8.21.1"
@ -965,6 +966,12 @@
"node": ">= 0.4"
}
},
"node_modules/hls.js": {
"version": "1.6.16",
"resolved": "https://registry.npmjs.org/hls.js/-/hls.js-1.6.16.tgz",
"integrity": "sha512-VSIRpLfRwlAAdGL4wiTucx2ScRipo0ed1FBatWkyt832jC4CReKstga6yIhYVwGu9LOBjuX9wzmRMeQdBJtzEA==",
"license": "Apache-2.0"
},
"node_modules/http-errors": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz",

View File

@ -1,6 +1,6 @@
{
"name": "lumi-bot",
"version": "0.2.27",
"version": "0.3.11",
"private": true,
"type": "commonjs",
"scripts": {
@ -16,6 +16,7 @@
"verify:web-auth": "node scripts/verify-web-auth.js",
"verify:destructive-actions": "node scripts/verify-destructive-actions.js",
"verify:overlays": "node scripts/verify-overlays.js",
"verify:stream-testing": "node scripts/verify-stream-testing.js",
"verify:ux": "node scripts/verify-ux-foundation.js",
"test:ui": "playwright test",
"test:ui:update": "playwright test --update-snapshots",
@ -23,7 +24,10 @@
"benchmark:okf": "node scripts/benchmark-okf-search.js",
"verify:content": "node scripts/verify-content-library.js",
"verify:transcription": "node plugins/lumi_transcription/tests/verify.js",
"verify:dev-updates": "node scripts/verify-local-development-updates.js"
"verify:companion-overlay": "node plugins/lumi_overlay/tests/verify.js",
"verify:auto-vc": "node plugins/auto-vc/tests/verify.js",
"verify:dev-updates": "node scripts/verify-local-development-updates.js",
"verify:user-age": "node scripts/verify-user-age-statistics.js"
},
"engines": {
"node": ">=18"
@ -36,6 +40,7 @@
"ejs": "^3.1.10",
"express": "^4.19.2",
"express-session": "^1.18.1",
"hls.js": "^1.6.16",
"multer": "^2.2.0",
"tmi.js": "^1.8.5",
"ws": "^8.21.1"

View File

@ -1,5 +1,17 @@
# Auto VC changelog
## 0.1.7
- Add an admin-defined message to each new room with the
`{{plugins.auto_vc.channel.owner_username}}` placeholder.
- Add persistent lock and unlock reactions that reuse the existing room
permission behavior.
- Add a claim reaction after the owner has been absent for the configured
cleanup duration, plus concise success or failure reports that delete after
30 seconds.
- Preserve existing room and lobby data with additive control-message and
owner-absence columns.
## 0.1.6
- Connected lobby deletion to the shared timed-confirmation flow.

View File

@ -3,10 +3,19 @@ const fs = require("fs");
const ejs = require("ejs");
const { Permissions } = require("discord.js");
const { ensureUserForIdentity } = require("../../src/services/users");
const { createLogger } = require("../../src/services/logger");
const logger = createLogger("plugin:auto-vc", { category: "plugin" });
const PLUGIN_ID = "auto-vc";
const DEFAULT_TEMPLATE = "[username]'s room";
const DEFAULT_TIMEOUT = 30;
const DEFAULT_ROOM_MESSAGE = "Welcome {{plugins.auto_vc.channel.owner_username}}! Use 🔒 to lock this room or 🔓 to unlock it. If the owner is away long enough, 👑 appears so someone in the room can claim it.";
const ROOM_MESSAGE_FIELD_ID = "plugins.auto_vc.room_message";
const LOCK_EMOJI = "🔒";
const UNLOCK_EMOJI = "🔓";
const CLAIM_EMOJI = "👑";
const REPORT_LIFETIME_MS = 30 * 1000;
const GAME_NAME_TOKEN = "[game_name]";
const NAME_REFRESH_INTERVAL_MS = 5 * 60 * 1000;
const ALLOW_CONNECT_VIEW =
@ -16,12 +25,16 @@ const DEFAULT_ACTION_LIMIT = { max: 8, windowSeconds: 60 };
module.exports = {
id: PLUGIN_ID,
init({ web, discordClient, db, settings }) {
init({ web, discordClient, db, settings, placeholders }) {
ensureTables(db);
registerPlaceholderSupport(placeholders);
const state = {
rooms: new Map(),
cleanupTimers: new Map(),
claimTimers: new Map(),
emptySince: new Map(),
reactionActions: new Map(),
placeholders,
sweepTimer: null,
nameSweepTimer: null,
rateLimits: {
@ -63,6 +76,7 @@ module.exports = {
}
const config = parseConfigFromForm(req.body);
saveConfig(db, config);
for (const channelId of state.claimTimers.keys()) clearClaimTimer(state, channelId);
state.rooms.clear();
req.session.flash = {
type: "success",
@ -127,7 +141,7 @@ module.exports = {
const attach = () => {
bootstrapRooms(discordClient, db, state, settings).catch((error) => {
console.error("Auto VC bootstrap failed", error);
logger.error("Auto VC bootstrap failed", error, { event: "bootstrap_failed" });
});
discordClient.on("voiceStateUpdate", (oldState, newState) => {
handleVoiceStateUpdate(oldState, newState, db, settings, state);
@ -135,6 +149,9 @@ module.exports = {
discordClient.on("messageCreate", (message) => {
handleMessage(message, db, settings, state);
});
discordClient.on("messageReactionAdd", (reaction, user) => {
queueControlReaction(reaction, user, db, settings, state);
});
discordClient.on("channelDelete", (channel) => {
if (channel && channel.id) {
removeRoom(db, state, channel.id);
@ -215,7 +232,9 @@ async function buildLobbyPermissionChecks(discordClient, lobby) {
"Connect to lobby",
"Move members",
"Target category visible",
"Manage rooms"
"Manage rooms",
"Post room controls",
"Manage room reactions"
];
if (!discordClient || !discordClient.user) {
@ -268,6 +287,22 @@ async function buildLobbyPermissionChecks(discordClient, lobby) {
lobbyPerms?.has(Permissions.FLAGS.MANAGE_CHANNELS) ||
guildPerms?.has(Permissions.FLAGS.MANAGE_CHANNELS)
);
const canSendMessages = Boolean(
categoryPerms?.has(Permissions.FLAGS.SEND_MESSAGES) ||
guildPerms?.has(Permissions.FLAGS.SEND_MESSAGES)
);
const canAddReactions = Boolean(
categoryPerms?.has(Permissions.FLAGS.ADD_REACTIONS) ||
guildPerms?.has(Permissions.FLAGS.ADD_REACTIONS)
);
const canReadHistory = Boolean(
categoryPerms?.has(Permissions.FLAGS.READ_MESSAGE_HISTORY) ||
guildPerms?.has(Permissions.FLAGS.READ_MESSAGE_HISTORY)
);
const canManageMessages = Boolean(
categoryPerms?.has(Permissions.FLAGS.MANAGE_MESSAGES) ||
guildPerms?.has(Permissions.FLAGS.MANAGE_MESSAGES)
);
return [
buildPermissionCheck("Bot in guild", true, ""),
@ -304,6 +339,16 @@ async function buildLobbyPermissionChecks(discordClient, lobby) {
"Manage rooms",
canManageChannels && (categoryChannel ? canViewCategory : true),
"Allow Manage Channels (and View Channel) on the target category so the bot can create, rename, and delete rooms."
),
buildPermissionCheck(
"Post room controls",
canSendMessages && canAddReactions && canReadHistory,
"Allow Send Messages, Add Reactions, and Read Message History on the target category."
),
buildPermissionCheck(
"Manage room reactions",
canManageMessages,
"Allow Manage Messages on the target category so reaction controls remain reusable."
)
];
}
@ -358,6 +403,8 @@ function ensureTables(db) {
locked INTEGER NOT NULL DEFAULT 0,
allowed_user_ids TEXT NOT NULL DEFAULT '[]',
base_overwrites TEXT NOT NULL,
control_message_id TEXT,
owner_absent_since INTEGER,
created_at INTEGER NOT NULL
);
@ -373,6 +420,15 @@ function ensureTables(db) {
created_at INTEGER NOT NULL
);
`);
ensureColumn(db, "auto_vc_rooms", "control_message_id", "TEXT");
ensureColumn(db, "auto_vc_rooms", "owner_absent_since", "INTEGER");
}
function ensureColumn(db, table, column, definition) {
const columns = db.prepare(`PRAGMA table_info(${table})`).all();
if (!columns.some((entry) => entry.name === column)) {
db.exec(`ALTER TABLE ${table} ADD COLUMN ${column} ${definition}`);
}
}
function getConfig(db) {
@ -405,6 +461,7 @@ function normalizeLobby(lobby) {
lobbyChannelId: (lobby?.lobbyChannelId || "").toString().trim(),
categoryId: (lobby?.categoryId || "").toString().trim(),
nameTemplate: (lobby?.nameTemplate || DEFAULT_TEMPLATE).toString(),
roomMessage: (lobby?.roomMessage ?? DEFAULT_ROOM_MESSAGE).toString().slice(0, 2000),
emptyTimeoutSeconds: Number.isNaN(timeout) ? DEFAULT_TIMEOUT : Math.max(5, timeout)
};
}
@ -448,6 +505,7 @@ function parseConfigFromForm(body) {
const lobbyChannelIds = toArray(body.lobby_channel_id);
const categoryIds = toArray(body.lobby_category_id);
const templates = toArray(body.lobby_name_template);
const roomMessages = toArray(body.lobby_room_message);
const timeouts = toArray(body.lobby_empty_timeout);
const removeIds = new Set(toArray(body.lobby_remove));
@ -458,6 +516,7 @@ function parseConfigFromForm(body) {
lobbyChannelId: lobbyChannelIds[index] || "",
categoryId: categoryIds[index] || "",
nameTemplate: templates[index] || DEFAULT_TEMPLATE,
roomMessage: roomMessages[index] ?? DEFAULT_ROOM_MESSAGE,
emptyTimeoutSeconds: Number.isNaN(timeout) ? DEFAULT_TIMEOUT : timeout
});
});
@ -531,6 +590,7 @@ function isBanned(db, discordUserId) {
}
async function bootstrapRooms(discordClient, db, state, settings) {
const config = getConfig(db);
const rooms = db
.prepare("SELECT * FROM auto_vc_rooms")
.all();
@ -551,7 +611,9 @@ async function bootstrapRooms(discordClient, db, state, settings) {
removeRoom(db, state, room.channel_id);
continue;
}
state.rooms.set(room.channel_id, normalizeRoom(room));
const normalized = normalizeRoom(room);
state.rooms.set(room.channel_id, normalized);
await syncOwnerAbsence(channel, normalized, db, state, config);
}
}
@ -568,6 +630,8 @@ function normalizeRoom(room) {
locked: Boolean(room.locked),
allowed_user_ids: parseJsonArray(room.allowed_user_ids),
base_overwrites: room.base_overwrites,
control_message_id: room.control_message_id || null,
owner_absent_since: Number(room.owner_absent_since) || null,
created_at: room.created_at
};
}
@ -580,7 +644,7 @@ function handleVoiceStateUpdate(oldState, newState, db, settings, state) {
if (lobby && newState.channelId !== oldState.channelId) {
createRoomFromLobby(newState, lobby, db, settings, state, config).catch((error) => {
console.error("Auto VC creation failed", error);
logger.error("Auto VC creation failed", error, { event: "room_creation_failed" });
});
}
@ -590,6 +654,9 @@ function handleVoiceStateUpdate(oldState, newState, db, settings, state) {
const channel = oldState.guild.channels.cache.get(oldState.channelId);
if (channel) {
scheduleCleanup(channel, room, db, state, config);
syncOwnerAbsence(channel, room, db, state, config).catch((error) => {
logger.warn("Auto VC owner absence update failed", error, { event: "owner_absence_sync_failed" });
});
}
}
}
@ -599,6 +666,12 @@ function handleVoiceStateUpdate(oldState, newState, db, settings, state) {
if (room) {
clearCleanupTimer(state, newState.channelId);
clearEmpty(state, newState.channelId);
const channel = newState.guild.channels.cache.get(newState.channelId);
if (channel) {
syncOwnerAbsence(channel, room, db, state, config).catch((error) => {
logger.warn("Auto VC owner absence update failed", error, { event: "owner_absence_sync_failed" });
});
}
}
}
}
@ -657,11 +730,14 @@ async function createRoomFromLobby(newState, lobby, db, settings, state, config)
locked: false,
allowed_user_ids: [],
base_overwrites: JSON.stringify(baseOverwrites),
control_message_id: null,
owner_absent_since: null,
created_at: Date.now()
};
saveRoom(db, room);
state.rooms.set(channel.id, normalizeRoom(room));
const normalizedRoom = normalizeRoom(room);
state.rooms.set(channel.id, normalizedRoom);
incrementUserStat(db, profile.id);
const moved = await moveMemberToChannel(member, channel);
@ -671,6 +747,83 @@ async function createRoomFromLobby(newState, lobby, db, settings, state, config)
"I couldn't move you to the new VC. Please make sure the bot has Move Members permission."
);
}
await createRoomControlMessage(channel, normalizedRoom, lobby, member, db, state);
}
function registerPlaceholderSupport(placeholders) {
if (!placeholders?.registerFieldPolicy || !placeholders?.registerPlaceholders) return;
placeholders.registerFieldPolicy({
field_id: ROOM_MESSAGE_FIELD_ID,
label: "Auto VC room message",
field_type: "chat_message",
output_audience: "public",
min_editor_role: "admin",
allowed_namespaces: ["plugins.auto_vc.channel"],
max_sensitivity: "public_safe"
});
placeholders.registerPlaceholders([{
id: "plugins.auto_vc.channel.owner_username",
namespace: "plugins.auto_vc.channel",
label: "Room owner username",
description: "The display name of the member who created the Auto VC.",
value_type: "string",
sensitivity: "public_safe",
min_editor_role: "admin",
min_viewer_role: "public",
allowed_field_types: ["chat_message"],
plugin_id: PLUGIN_ID,
resolver: ({ runtimeContext }) => runtimeContext?.autoVc?.owner_username || ""
}]);
}
async function renderRoomMessage(lobby, member, placeholders) {
const ownerUsername = member?.displayName || member?.user?.username || "Room owner";
const template = String(lobby?.roomMessage || DEFAULT_ROOM_MESSAGE).slice(0, 2000);
if (!placeholders?.renderTemplate) {
return template.replace(/\{\{\s*plugins\.auto_vc\.channel\.owner_username\s*\}\}/gi, ownerUsername).slice(0, 2000);
}
const result = await placeholders.renderTemplate({
fieldId: ROOM_MESSAGE_FIELD_ID,
template,
user: { isAdmin: true },
outputAudience: "public",
runtimeContext: { runtime: true, autoVc: { owner_username: ownerUsername } },
fallback: "[unavailable]"
});
if (!result.ok) {
logger.warn("Auto VC room message contained unavailable placeholders", {
error_count: result.errors?.length || 0
}, { event: "room_message_placeholder_failed" });
}
return String(result.rendered || DEFAULT_ROOM_MESSAGE).trim().slice(0, 2000);
}
async function createRoomControlMessage(channel, room, lobby, member, db, state) {
if (typeof channel?.send !== "function") {
logger.warn("Created Auto VC does not support room messages", {
channel_id: channel?.id || null
}, { event: "room_controls_unavailable" });
return null;
}
let message = null;
try {
const content = await renderRoomMessage(lobby, member, state.placeholders);
message = await channel.send(content || DEFAULT_ROOM_MESSAGE);
await message.react(LOCK_EMOJI);
await message.react(UNLOCK_EMOJI);
room.control_message_id = message.id;
room.owner_absent_since = null;
updateRoom(db, room);
state.rooms.set(room.channel_id, room);
return message;
} catch (error) {
if (message) await message.delete().catch(() => null);
logger.warn("Failed to create Auto VC room controls", error, {
event: "room_controls_create_failed",
channel_id: channel?.id || null
});
return null;
}
}
function buildRoomName(template, member, roomNumber, gameName) {
@ -759,9 +912,9 @@ function extractOverwrites(channel) {
function saveRoom(db, room) {
db.prepare(
"INSERT INTO auto_vc_rooms (channel_id, guild_id, lobby_id, category_id, owner_discord_id, owner_user_id, room_number, name_template, locked, allowed_user_ids, base_overwrites, created_at) " +
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) " +
"ON CONFLICT(channel_id) DO UPDATE SET owner_discord_id = excluded.owner_discord_id, owner_user_id = excluded.owner_user_id, room_number = excluded.room_number, name_template = excluded.name_template, locked = excluded.locked, allowed_user_ids = excluded.allowed_user_ids, base_overwrites = excluded.base_overwrites"
"INSERT INTO auto_vc_rooms (channel_id, guild_id, lobby_id, category_id, owner_discord_id, owner_user_id, room_number, name_template, locked, allowed_user_ids, base_overwrites, control_message_id, owner_absent_since, created_at) " +
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) " +
"ON CONFLICT(channel_id) DO UPDATE SET owner_discord_id = excluded.owner_discord_id, owner_user_id = excluded.owner_user_id, room_number = excluded.room_number, name_template = excluded.name_template, locked = excluded.locked, allowed_user_ids = excluded.allowed_user_ids, base_overwrites = excluded.base_overwrites, control_message_id = excluded.control_message_id, owner_absent_since = excluded.owner_absent_since"
).run(
room.channel_id,
room.guild_id,
@ -774,13 +927,15 @@ function saveRoom(db, room) {
room.locked ? 1 : 0,
JSON.stringify(room.allowed_user_ids || []),
room.base_overwrites,
room.control_message_id || null,
room.owner_absent_since || null,
room.created_at
);
}
function updateRoom(db, room) {
db.prepare(
"UPDATE auto_vc_rooms SET owner_discord_id = ?, owner_user_id = ?, room_number = ?, name_template = ?, locked = ?, allowed_user_ids = ? WHERE channel_id = ?"
"UPDATE auto_vc_rooms SET owner_discord_id = ?, owner_user_id = ?, room_number = ?, name_template = ?, locked = ?, allowed_user_ids = ?, control_message_id = ?, owner_absent_since = ? WHERE channel_id = ?"
).run(
room.owner_discord_id,
room.owner_user_id,
@ -788,6 +943,8 @@ function updateRoom(db, room) {
room.name_template,
room.locked ? 1 : 0,
JSON.stringify(room.allowed_user_ids || []),
room.control_message_id || null,
room.owner_absent_since || null,
room.channel_id
);
}
@ -806,6 +963,7 @@ function getRoomById(db, channelId, state) {
function removeRoom(db, state, channelId) {
clearCleanupTimer(state, channelId);
clearClaimTimer(state, channelId);
clearEmpty(state, channelId);
state.rooms.delete(channelId);
db.prepare("DELETE FROM auto_vc_rooms WHERE channel_id = ?").run(channelId);
@ -919,11 +1077,173 @@ function getLobbyTimeout(config, lobbyId) {
return lobby ? lobby.emptyTimeoutSeconds : DEFAULT_TIMEOUT;
}
function clearClaimTimer(state, channelId) {
const timer = state.claimTimers.get(channelId);
if (timer) {
clearTimeout(timer);
state.claimTimers.delete(channelId);
}
}
function hasClaimCandidate(channel, room) {
return [...(channel?.members?.values?.() || [])].some((member) =>
!member?.user?.bot && member.id !== room.owner_discord_id
);
}
async function fetchControlMessage(channel, room) {
if (!room.control_message_id || typeof channel?.messages?.fetch !== "function") return null;
return channel.messages.fetch(room.control_message_id).catch(() => null);
}
async function setClaimReaction(channel, room, available) {
const message = await fetchControlMessage(channel, room);
if (!message) return;
const reaction = message.reactions?.cache?.find((entry) => entry.emoji?.name === CLAIM_EMOJI);
if (available) {
if (!reaction?.me) await message.react(CLAIM_EMOJI);
return;
}
if (reaction?.me && message.client?.user?.id) {
await reaction.users.remove(message.client.user.id).catch(() => null);
}
}
async function syncOwnerAbsence(channel, room, db, state, config) {
clearClaimTimer(state, channel.id);
const ownerPresent = Boolean(channel.members?.has(room.owner_discord_id));
const claimCandidatePresent = hasClaimCandidate(channel, room);
if (ownerPresent || !claimCandidatePresent) {
if (room.owner_absent_since) {
room.owner_absent_since = null;
updateRoom(db, room);
state.rooms.set(room.channel_id, room);
}
await setClaimReaction(channel, room, false);
return;
}
if (!room.owner_absent_since) {
room.owner_absent_since = Date.now();
updateRoom(db, room);
state.rooms.set(room.channel_id, room);
}
const timeoutMs = getLobbyTimeout(config, room.lobby_id) * 1000;
const remaining = timeoutMs - (Date.now() - room.owner_absent_since);
if (remaining <= 0) {
await setClaimReaction(channel, room, true);
return;
}
await setClaimReaction(channel, room, false);
const timer = setTimeout(() => {
state.claimTimers.delete(channel.id);
const refreshedRoom = state.rooms.get(channel.id) || getRoomById(db, channel.id, state);
const refreshedChannel = channel.guild?.channels?.cache?.get(channel.id);
if (!refreshedRoom || !refreshedChannel) return;
syncOwnerAbsence(refreshedChannel, refreshedRoom, db, state, getConfig(db)).catch((error) => {
logger.warn("Auto VC claim reaction update failed", error, { event: "claim_reaction_update_failed" });
});
}, remaining);
timer.unref?.();
state.claimTimers.set(channel.id, timer);
}
function queueControlReaction(reaction, user, db, settings, state) {
if (!reaction?.message?.id || user?.bot) return;
const key = reaction.message.id;
const previous = state.reactionActions.get(key) || Promise.resolve();
const current = previous
.catch(() => null)
.then(() => handleControlReaction(reaction, user, db, settings, state))
.catch((error) => {
logger.warn("Auto VC reaction action failed", error, { event: "room_reaction_failed" });
})
.finally(() => {
if (state.reactionActions.get(key) === current) state.reactionActions.delete(key);
});
state.reactionActions.set(key, current);
}
async function handleControlReaction(reaction, user, db, settings, state) {
if (reaction.partial && typeof reaction.fetch === "function") {
reaction = await reaction.fetch().catch(() => null);
}
if (!reaction?.message) return;
let message = reaction.message;
if (message.partial && typeof message.fetch === "function") {
message = await message.fetch().catch(() => null);
}
const emoji = reaction.emoji?.name;
if (!message || ![LOCK_EMOJI, UNLOCK_EMOJI, CLAIM_EMOJI].includes(emoji)) return;
const channel = message.channel;
const room = state.rooms.get(channel?.id) || getRoomById(db, channel?.id, state);
if (!room || room.control_message_id !== message.id) return;
try {
const member = await message.guild?.members?.fetch(user.id).catch(() => null);
let result;
if (!member) {
result = { ok: false, message: "Could not verify that member." };
} else {
const rateLimit = consumeRateLimit(state, getConfig(db), "action", member.id);
if (!rateLimit.ok) {
result = { ok: false, message: `Try again in ${formatCooldown(rateLimit.retryAfter)}.` };
} else if (emoji === LOCK_EMOJI || emoji === UNLOCK_EMOJI) {
if (member.id !== room.owner_discord_id) {
result = { ok: false, message: "Only the room owner can use that control." };
} else {
const locked = emoji === LOCK_EMOJI;
const applied = await setRoomLocked(channel, room, locked, db, settings, state);
result = applied
? { ok: true, message: locked ? "Room locked." : "Room unlocked." }
: { ok: false, message: "Discord did not allow the room permission change." };
}
} else {
const eligibility = claimEligibility(channel, room, member, getConfig(db), Date.now());
if (!eligibility.ok) {
result = eligibility;
} else {
const claimed = await claimRoomOwnership(channel, room, member, db, settings, state);
if (claimed) {
await syncOwnerAbsence(channel, room, db, state, getConfig(db));
result = { ok: true, message: `${member.displayName} now owns this room.` };
} else {
result = { ok: false, message: "Discord did not allow the ownership change." };
}
}
}
}
await sendTemporaryReport(channel, `${result.ok ? "✓" : "Couldnt do that:"} ${result.message}`);
} finally {
await reaction.users?.remove(user.id).catch(() => null);
}
}
function claimEligibility(channel, room, member, config, now = Date.now()) {
if (member.id === room.owner_discord_id) return { ok: false, message: "You already own this room." };
if (!channel.members?.has(member.id)) return { ok: false, message: "Join this voice room before claiming it." };
if (channel.members?.has(room.owner_discord_id)) return { ok: false, message: "The current owner is still here." };
const absentSince = Number(room.owner_absent_since) || now;
const remaining = getLobbyTimeout(config, room.lobby_id) * 1000 - (now - absentSince);
if (remaining > 0) return { ok: false, message: `The room can be claimed in ${formatCooldown(Math.ceil(remaining / 1000))}.` };
return { ok: true };
}
async function sendTemporaryReport(channel, content) {
if (typeof channel?.send !== "function") return;
const report = await channel.send(String(content).slice(0, 500)).catch(() => null);
if (!report) return;
const timer = setTimeout(() => {
report.delete().catch(() => null);
}, REPORT_LIFETIME_MS);
timer.unref?.();
}
async function deleteChannel(channel) {
try {
await channel.delete("Auto VC cleanup");
} catch (error) {
console.error("Failed to delete Auto VC channel", error);
logger.error("Failed to delete Auto VC channel", error, { event: "channel_delete_failed" });
}
}
@ -1061,13 +1381,12 @@ async function handleLock(message, room, db, settings, state, config) {
if (!enforceActionRateLimit(message, state, config)) {
return;
}
room.locked = !room.locked;
await applyRoomPermissions(channel, room, db, settings);
updateRoom(db, room);
state.rooms.set(room.channel_id, room);
message
.reply(room.locked ? "Room locked." : "Room unlocked.")
.catch(() => null);
if (room.locked) {
message.reply("Room is already locked.").catch(() => null);
return;
}
const applied = await setRoomLocked(channel, room, true, db, settings, state);
message.reply(applied ? "Room locked." : "I couldn't update this room's permissions.").catch(() => null);
}
async function handleUnlock(message, room, db, settings, state, config) {
@ -1083,11 +1402,20 @@ async function handleUnlock(message, room, db, settings, state, config) {
message.reply("Room is already unlocked.").catch(() => null);
return;
}
room.locked = false;
await applyRoomPermissions(channel, room, db, settings);
const applied = await setRoomLocked(channel, room, false, db, settings, state);
message.reply(applied ? "Room unlocked." : "I couldn't update this room's permissions.").catch(() => null);
}
async function setRoomLocked(channel, room, locked, db, settings, state) {
const previous = room.locked;
room.locked = Boolean(locked);
if (!await applyRoomPermissions(channel, room, db, settings)) {
room.locked = previous;
return false;
}
updateRoom(db, room);
state.rooms.set(room.channel_id, room);
message.reply("Room unlocked.").catch(() => null);
return true;
}
async function handleAllow(message, room, args, db, settings, state, config) {
@ -1165,12 +1493,15 @@ async function handleTransfer(message, room, args, db, settings, state, config)
displayName: target.displayName
});
room.owner_user_id = profile.id;
room.owner_absent_since = null;
const channel = message.guild.channels.cache.get(room.channel_id);
if (channel) {
await applyRoomPermissions(channel, room, db, settings);
}
updateRoom(db, room);
state.rooms.set(room.channel_id, room);
clearClaimTimer(state, room.channel_id);
if (channel) await setClaimReaction(channel, room, false);
message.reply(`Ownership transferred to ${target.displayName}.`).catch(() => null);
}
@ -1187,6 +1518,14 @@ async function handleClaim(message, room, member, db, settings, state, config) {
if (!enforceActionRateLimit(message, state, config)) {
return;
}
const claimed = await claimRoomOwnership(channel, room, member, db, settings, state);
message.reply(claimed ? "You are now the owner of this room." : "I couldn't update this room's permissions.").catch(() => null);
}
async function claimRoomOwnership(channel, room, member, db, settings, state) {
const previousOwnerDiscordId = room.owner_discord_id;
const previousOwnerUserId = room.owner_user_id;
const previousOwnerAbsentSince = room.owner_absent_since;
room.owner_discord_id = member.id;
const profile = ensureUserForIdentity({
provider: "discord",
@ -1194,10 +1533,18 @@ async function handleClaim(message, room, member, db, settings, state, config) {
displayName: member.displayName
});
room.owner_user_id = profile.id;
await applyRoomPermissions(channel, room, db, settings);
room.owner_absent_since = null;
if (!await applyRoomPermissions(channel, room, db, settings)) {
room.owner_discord_id = previousOwnerDiscordId;
room.owner_user_id = previousOwnerUserId;
room.owner_absent_since = previousOwnerAbsentSince;
return false;
}
updateRoom(db, room);
state.rooms.set(room.channel_id, room);
message.reply("You are now the owner of this room.").catch(() => null);
clearClaimTimer(state, room.channel_id);
await setClaimReaction(channel, room, false);
return true;
}
async function applyRoomPermissions(channel, room, db, settings) {
@ -1246,7 +1593,16 @@ async function applyRoomPermissions(channel, room, db, settings) {
}
}
await channel.permissionOverwrites.set(Array.from(overrides.values())).catch(() => null);
try {
await channel.permissionOverwrites.set(Array.from(overrides.values()));
return true;
} catch (error) {
logger.warn("Failed to update Auto VC permissions", error, {
event: "room_permissions_update_failed",
channel_id: channel?.id || null
});
return false;
}
}
function checkIsModerator(member, settings) {
@ -1348,6 +1704,21 @@ function cryptoRandomId() {
return require("crypto").randomUUID();
}
module.exports._test = {
CLAIM_EMOJI,
DEFAULT_ROOM_MESSAGE,
LOCK_EMOJI,
UNLOCK_EMOJI,
claimEligibility,
ensureTables,
normalizeConfig,
normalizeLobby,
parseConfigFromForm,
registerPlaceholderSupport,
renderRoomMessage,
setRoomLocked
};
async function safeNotify(member, message) {
try {
await member.send(message);
@ -1361,12 +1732,12 @@ async function moveMemberToChannel(member, channel) {
await member.voice.setChannel(channel);
return true;
} catch (error) {
console.error("Failed to move member", error);
logger.error("Failed to move member", error, { event: "member_move_failed" });
try {
await member.edit({ channel: channel.id });
return true;
} catch (fallbackError) {
console.error("Fallback move failed", fallbackError);
logger.error("Fallback move failed", fallbackError, { event: "member_move_fallback_failed" });
return false;
}
}
@ -1378,7 +1749,7 @@ function startNameRefreshTimer(discordClient, db, settings, state) {
}
const runSweep = () => {
refreshRoomNames(discordClient, db, settings, state).catch((error) => {
console.error("Auto VC name refresh failed", error);
logger.error("Auto VC name refresh failed", error, { event: "name_refresh_failed" });
});
};
runSweep();
@ -1408,7 +1779,7 @@ async function refreshRoomNames(discordClient, db, settings, state) {
const desiredName = buildRoomName(room.name_template, member, room.room_number, gameName);
if (desiredName && desiredName !== channel.name) {
await channel.setName(desiredName).catch((error) => {
console.error("Failed to update Auto VC name", error);
logger.error("Failed to update Auto VC name", error, { event: "channel_rename_failed" });
});
}
}
@ -1420,7 +1791,7 @@ function startSweepTimer(discordClient, db, settings, state) {
}
const runSweep = () => {
sweepRooms(discordClient, db, state).catch((error) => {
console.error("Auto VC sweep failed", error);
logger.error("Auto VC sweep failed", error, { event: "room_sweep_failed" });
});
};
runSweep();
@ -1443,6 +1814,7 @@ async function sweepRooms(discordClient, db, state) {
removeRoom(db, state, room.channel_id);
continue;
}
await syncOwnerAbsence(channel, room, db, state, config);
if (channel.members.size === 0) {
markEmpty(state, channel.id);
const timeout = getLobbyTimeout(config, room.lobby_id);

View File

@ -1,11 +1,11 @@
{
"id": "auto-vc",
"name": "Auto VC",
"version": "0.1.6",
"version": "0.1.7",
"description": "Auto-create managed voice channels from lobby rooms.",
"main": "index.js",
"channel": "stable",
"compatible_from": "0.1.5",
"migration_notes": "No manual migration is required.",
"migration_notes": "Adds persisted room-control messages and owner-absence timestamps with automatic additive database migration. Existing lobbies, rooms, owners, permissions, bans, and statistics remain preserved.",
"rollback_safe": true
}

View File

@ -0,0 +1,105 @@
"use strict";
const assert = require("assert");
const Database = require("better-sqlite3");
const plugin = require("../index");
async function run() {
const {
CLAIM_EMOJI,
DEFAULT_ROOM_MESSAGE,
LOCK_EMOJI,
UNLOCK_EMOJI,
claimEligibility,
ensureTables,
normalizeConfig,
parseConfigFromForm,
registerPlaceholderSupport,
renderRoomMessage,
setRoomLocked
} = plugin._test;
const defaults = normalizeConfig({ lobbies: [{ id: "lobby-1" }] });
assert.equal(defaults.lobbies[0].roomMessage, DEFAULT_ROOM_MESSAGE);
assert.equal(defaults.lobbies[0].emptyTimeoutSeconds, 30);
const parsed = parseConfigFromForm({
lobby_id: ["one", "two"],
lobby_channel_id: ["voice-1", "voice-2"],
lobby_category_id: ["category-1", "category-2"],
lobby_name_template: ["[username] one", "[username] two"],
lobby_room_message: ["Hello one", "Hello two"],
lobby_empty_timeout: ["45", "60"]
});
assert.deepStrictEqual(parsed.lobbies.map((lobby) => lobby.roomMessage), ["Hello one", "Hello two"]);
assert.deepStrictEqual(parsed.lobbies.map((lobby) => lobby.emptyTimeoutSeconds), [45, 60]);
const database = new Database(":memory:");
database.exec("CREATE TABLE auto_vc_rooms (channel_id TEXT PRIMARY KEY)");
ensureTables(database);
const columns = database.prepare("PRAGMA table_info(auto_vc_rooms)").all().map((column) => column.name);
assert(columns.includes("control_message_id"), "room controls must survive a restart");
assert(columns.includes("owner_absent_since"), "owner absence must survive a restart");
database.close();
let policy;
let definitions;
const placeholders = {
registerFieldPolicy(value) { policy = value; },
registerPlaceholders(value) { definitions = value; },
async renderTemplate({ template, runtimeContext }) {
return {
ok: true,
rendered: template.replace("{{plugins.auto_vc.channel.owner_username}}", runtimeContext.autoVc.owner_username)
};
}
};
registerPlaceholderSupport(placeholders);
assert.equal(policy.field_id, "plugins.auto_vc.room_message");
assert(policy.allowed_namespaces.includes("plugins.auto_vc.channel"));
assert.equal(definitions[0].id, "plugins.auto_vc.channel.owner_username");
const rendered = await renderRoomMessage(
{ roomMessage: "Welcome {{plugins.auto_vc.channel.owner_username}}" },
{ displayName: "Lumi User" },
placeholders
);
assert.equal(rendered, "Welcome Lumi User");
const members = new Map([["claimant", { id: "claimant", user: { bot: false } }]]);
const channel = { members };
const config = { lobbies: [{ id: "lobby-1", emptyTimeoutSeconds: 30 }] };
const room = {
lobby_id: "lobby-1",
owner_discord_id: "owner",
owner_absent_since: 1_000
};
assert.equal(claimEligibility(channel, room, { id: "claimant" }, config, 30_999).ok, false);
assert.equal(claimEligibility(channel, room, { id: "claimant" }, config, 31_000).ok, true);
members.set("owner", { id: "owner", user: { bot: false } });
assert.equal(claimEligibility(channel, room, { id: "claimant" }, config, 60_000).ok, false);
assert.deepStrictEqual([LOCK_EMOJI, UNLOCK_EMOJI, CLAIM_EMOJI], ["🔒", "🔓", "👑"]);
const failedRoom = {
channel_id: "room",
owner_discord_id: "owner",
locked: false,
allowed_user_ids: [],
base_overwrites: "[]"
};
const applied = await setRoomLocked({
id: "room",
guild: { roles: { everyone: { id: "everyone" } } },
permissionOverwrites: { async set() { throw new Error("missing permission"); } }
}, failedRoom, true, {
prepare() { throw new Error("a failed Discord change must not be persisted"); }
}, { getSetting() { return ""; } }, { rooms: new Map() });
assert.equal(applied, false);
assert.equal(failedRoom.locked, false, "failed permission changes must not report or persist success");
console.log("Auto VC verification passed: welcome placeholders, persisted controls, and timed ownership claims.");
}
run().catch((error) => {
console.error(error);
process.exitCode = 1;
});

View File

@ -275,6 +275,15 @@
<label>Empty room cleanup (seconds)</label>
<input name="lobby_empty_timeout" value="<%= lobby.emptyTimeoutSeconds %>" type="number" min="5" />
</div>
<div class="field full">
<label>New room message</label>
<textarea name="lobby_room_message" rows="4" maxlength="2000"><%= lobby.roomMessage %></textarea>
<p class="hint">
Sent in every newly created room with lock and unlock reactions.
Supports <code>{{plugins.auto_vc.channel.owner_username}}</code>.
The claim reaction appears after the owner has been away for the cleanup duration.
</p>
</div>
</div>
<% if (lobby.permissions && lobby.permissions.length) { %>
<% const totalPerms = lobby.permissions.length; %>
@ -428,6 +437,13 @@
<label>Empty room cleanup (seconds)</label>
<input name="lobby_empty_timeout" value="30" type="number" min="5" />
</div>
<div class="field full">
<label>New room message</label>
<textarea name="lobby_room_message" rows="4" maxlength="2000">Welcome {{plugins.auto_vc.channel.owner_username}}! Use 🔒 to lock this room or 🔓 to unlock it. If the owner is away long enough, 👑 appears so someone in the room can claim it.</textarea>
<p class="hint">
Sent with lock and unlock reactions. The claim reaction appears after the owner has been away for the cleanup duration.
</p>
</div>
</div>
</div>
</template>

View File

@ -1,5 +1,8 @@
const crypto = require("crypto");
const path = require("path");
const { createLogger } = require("../../src/services/logger");
const logger = createLogger("plugin:birthday", { category: "plugin" });
const PLUGIN_ID = "birthday";
const TIMER_KEY = Symbol.for("lumi.birthday.interval");
@ -539,9 +542,13 @@ function restartScheduler({ db, discordClient }) {
if (!config.enabled) {
return;
}
checkBirthdays({ db, discordClient }).catch((error) => console.error("Birthday check failed", error));
checkBirthdays({ db, discordClient }).catch((error) =>
logger.error("Birthday check failed", error, { event: "birthday_check_failed" })
);
global[TIMER_KEY] = setInterval(() => {
checkBirthdays({ db, discordClient }).catch((error) => console.error("Birthday check failed", error));
checkBirthdays({ db, discordClient }).catch((error) =>
logger.error("Birthday check failed", error, { event: "birthday_check_failed" })
);
}, config.birthday_check_interval_minutes * 60 * 1000);
}

View File

@ -6,11 +6,14 @@ const express = require("express");
const multer = require("multer");
const EventEmitter = require("events");
const { ensureUserForIdentity } = require("../../src/services/users");
const { createLogger } = require("../../src/services/logger");
const {
cleanupUploadedFiles,
validateUploadedFile
} = require("../../src/services/upload-security");
const logger = createLogger("plugin:economy-framework", { category: "plugin" });
const PLUGIN_ID = "economy-framework";
const LEGACY_STEM = ["echo", "nomy"].join("");
const LEGACY_PLUGIN_ID = `${LEGACY_STEM}-framework`;
@ -1423,7 +1426,9 @@ function startActivityRewardFlusher(db) {
try {
flushActivityRewards(db);
} catch (error) {
console.error("Activity reward flush failed", error);
logger.error("Activity reward flush failed", error, {
event: "activity_reward_flush_failed"
});
}
}, 60 * 1000);
}
@ -1489,7 +1494,9 @@ function flushActivityRewards(db) {
"DELETE FROM economy_activity_reward_hourly WHERE user_id = ? AND hour_start = ?"
).run(group.userId, group.hourStart);
} catch (error) {
console.error("Failed to apply queued activity reward", error);
logger.error("Failed to apply queued activity reward", error, {
event: "activity_reward_apply_failed"
});
}
}
}

View File

@ -1,5 +1,8 @@
const fs = require("fs");
const path = require("path");
const { createLogger } = require("../../src/services/logger");
const logger = createLogger("plugin:expression-interaction", { category: "plugin" });
const DEFAULT_ACTIONS = [
{ id: "hug", verb: "hugs", past: "hugged" },
@ -672,7 +675,9 @@ function writeCommandsManifest(config) {
const target = path.join(pluginMeta.dir, "cmds.json");
fs.writeFileSync(target, JSON.stringify(manifest, null, 2), "utf8");
} catch (error) {
console.error("Failed to write expression command manifest", error);
logger.error("Failed to write expression command manifest", error, {
event: "command_manifest_write_failed"
});
}
}

View File

@ -2,6 +2,7 @@ const fs = require("fs");
const path = require("path");
const express = require("express");
const { writeJsonAtomicSync } = require("../../src/services/safe-files");
const { createLogger } = require("../../src/services/logger");
const { ensureDataDirs, resolveData } = require("./backend/paths");
const { getConfig, saveConfig, getRuntimeState } = require("./backend/config_manager");
const { detectHardware, estimateAllocation, performanceTuningHints } = require("./backend/hardware");
@ -42,6 +43,7 @@ const { formatBytes, bytesFromMb, sanityCheckSize } = require("./backend/size_ut
const { SOURCE_DEFAULTS } = require("./backend/controller");
const PLUGIN_ID = "lumi_ai";
const logger = createLogger(`plugin:${PLUGIN_ID}`, { category: "plugin" });
const TOKEN_PRESETS = Object.freeze([
{ label: "Tiny (256)", value: 256, description: "Small helper replies and minimal context." },
{ label: "Very small (512)", value: 512, description: "Short replies and low memory usage." },
@ -63,7 +65,11 @@ module.exports = {
ensureDataDirs();
if (!repoIndexer.loadIndex()) {
try { repoIndexer.refreshIndex(); }
catch (error) { console.warn("Lumi AI repository index initialization failed", error.message); }
catch (error) {
logger.warn("Lumi AI repository index initialization failed", error, {
event: "repository_index_initialization_failed"
});
}
}
let config = getConfig();
metrics.configureRetention(config.work_history_retention);
@ -374,7 +380,11 @@ module.exports = {
sanityCheckSize("Installed model", modelFileSize, 100 * 1024 ** 3),
sanityCheckSize("Estimated GPU memory", bytesFromMb(gpuAllocation.estimated_gpu_memory_mb), 100 * 1024 ** 3)
].filter((check) => !check.valid);
for (const diagnostic of sizeDiagnostics) console.warn(`Lumi AI size diagnostic: ${diagnostic.message}`);
for (const diagnostic of sizeDiagnostics) {
logger.warn("Lumi AI size diagnostic reported an invalid value", {
message: diagnostic.message
}, { event: "size_diagnostic_invalid" });
}
const models = modelManifest.models.map((model) => ({
...model,
downloaded: fs.existsSync(resolveData("models", model.filename)),
@ -1651,7 +1661,11 @@ module.exports = {
locals: { endpoint: `/plugins/${PLUGIN_ID}` }
});
} else {
console.warn("Lumi AI assistant panel hook is unavailable; settings remain accessible.");
logger.warn(
"Lumi AI assistant panel hook is unavailable; settings remain accessible",
null,
{ event: "assistant_panel_hook_unavailable" }
);
}
ensureSidebarNavItem(settings);
registerAssistantCommands({
@ -1666,17 +1680,25 @@ module.exports = {
});
writeCommandsManifest(plugin?.dir || __dirname, config);
setImmediate(() => toolManager.loadEnabled().catch((error) =>
console.error("Lumi AI tool loader failed", error)
logger.error("Lumi AI tool loader failed", error, {
event: "tool_loader_failed"
})
));
if (config.enabled) {
setImmediate(() => ensureGateRuntime().catch((error) =>
console.error("Lumi AI gate runtime start failed", error)
logger.error("Lumi AI gate runtime start failed", error, {
event: "gate_runtime_start_failed"
})
));
}
const state = getRuntimeState();
if (shouldAutoResume(config, state)) {
setImmediate(() => startRuntimes({ resume: true }).catch((error) => console.error("Lumi AI runtime resume failed", error)));
setImmediate(() => startRuntimes({ resume: true }).catch((error) =>
logger.error("Lumi AI runtime resume failed", error, {
event: "runtime_resume_failed"
})
));
}
return async () => {
@ -1799,13 +1821,13 @@ function deniedImprovement(req, res) {
return sendImprovementError(req, res, 403, "Improvement Center access is not enabled for this account.");
}
function logImprovementActionError(req, error) {
console.error("Lumi AI feedback action failed", {
logger.error("Lumi AI feedback action failed", {
review_id: cleanText(req.params?.id, 100),
action: cleanText(req.body?.action, 30),
user_id: cleanText(req.session?.user?.id, 100),
code: cleanText(error?.code, 40),
message: cleanText(error?.message || error, 500)
});
error
}, { event: "feedback_action_failed" });
}
function parseIdList(value) {
return [...new Set(String(value || "")

View File

@ -0,0 +1,15 @@
# Lumi Overlay Feed changelog
## 0.1.1
- Preserve Discord emotes and GIF media in the authenticated Companion feed.
- Preserve Twitch badge artwork and all normalized HTTPS emote sources while
retaining the existing bounded payload and server-side filtering rules.
## 0.1.0
- Add the paired-device `overlay.read.v1` live feed for the native Companion
monitor overlay.
- Reuse Lumi's normalized Twitch, YouTube, and Discord chat/event streams with
server-side source filtering, bounded queues, sanitization, deduplication,
live-state catalog updates, and no history replay.

View File

@ -0,0 +1,150 @@
"use strict";
const { WebSocketServer, WebSocket } = require("ws");
const { isSecureRequest, requestOrigin } = require("../../../src/services/proxy-security");
const { insecureDeviceAllowed } = require("../../lumi_transcription/backend/companion/device_store");
const { onOverlayChatMessage } = require("../../../src/services/overlay-chat");
const { onLumiEvent } = require("../../../src/services/lumi-events");
const { parseClientMessage, sanitizeChat, sanitizeEvent, MAX_MESSAGE_BYTES } = require("./protocol");
const { listSources, listStatuses, sourceIdForChat, sourceIdForEvent, listEventTypes } = require("./sources");
const MAX_QUEUE = 300;
const MAX_DEDUPE = 2000;
class OverlayFeed {
constructor({ devices, logger }) {
this.devices = devices;
this.logger = logger || { info() {}, warn() {}, error() {} };
this.wss = new WebSocketServer({ noServer: true, maxPayload: MAX_MESSAGE_BYTES, perMessageDeflate: false });
this.sessions = new Set();
this.publishedSeen = new Map();
this.sequence = 0;
this.stopChat = onOverlayChatMessage((message) => this.publishChat(message));
this.stopEvent = onLumiEvent((event) => this.publishEvent(event));
this.wss.on("connection", (socket, request, device) => this.connected(socket, request, device));
}
upgrade(request, socket, head) {
const auth = this.devices.authenticate(request.headers.authorization, "overlay.read.v1");
if (!auth.allowed) return reject(socket, auth.reason === "capability_revoked" ? 403 : 401, auth.reason);
let origin;
try { origin = requestOrigin(request); } catch { return reject(socket, 400, "invalid_host"); }
if (!isSecureRequest(request) && !insecureDeviceAllowed(auth.device, origin, request.socket.remoteAddress))
return reject(socket, 426, "tls_required");
this.wss.handleUpgrade(request, socket, head, (ws) => this.wss.emit("connection", ws, request, auth.device));
}
connected(socket, _request, device) {
const session = {
socket, device, sources: new Set(), events: new Set(), subscribed: false,
queue: [], draining: false, seen: new Map(), lastPong: Date.now()
};
this.sessions.add(session);
this.send(session, "hello", {
protocol_version: 1,
capabilities: ["overlay.feed.v1", "sources.catalog.v1", "live-state.v1"],
sources: listSources(),
event_types: listEventTypes(),
statuses: listStatuses(),
heartbeat_ms: 15000
});
const heartbeat = setInterval(() => {
if (Date.now() - session.lastPong > 45000) return socket.close(4408, "heartbeat_timeout");
this.send(session, "ping", { at: Date.now() });
this.send(session, "catalog", { sources: listSources(), statuses: listStatuses() });
}, 15000);
heartbeat.unref?.();
socket.on("message", (raw, binary) => {
if (binary) return socket.close(4400, "text_messages_only");
try {
const message = parseClientMessage(raw);
if (message.type === "subscribe") {
session.sources = new Set(message.sources);
session.events = new Set(message.events);
session.subscribed = true;
this.send(session, "subscribed", { sources: [...session.sources], events: [...session.events] });
} else if (message.type === "pong") session.lastPong = Date.now();
else if (message.type === "ping") this.send(session, "pong", { at: Date.now() });
} catch (error) {
this.send(session, "error", { code: error.code || "INVALID_MESSAGE", message: error.message });
}
});
socket.on("close", () => { clearInterval(heartbeat); this.sessions.delete(session); });
socket.on("error", (error) => this.logger.warn?.("Native overlay feed connection failed", { device_id: device.id, error: error.message }, { event: "overlay_feed_socket_error" }));
}
publishChat(message) {
const sourceId = sourceIdForChat(message);
if (!this.acceptPublished(`chat:${message.id}`)) return;
for (const session of this.sessions) {
if (!session.subscribed || !session.sources.has(sourceId)) continue;
this.enqueue(session, "chat", { source_id: sourceId, message: sanitizeChat(message) }, `chat:${message.id}`);
}
}
publishEvent(event) {
const sourceId = sourceIdForEvent(event);
if (!this.acceptPublished(`event:${event.id}`)) return;
for (const session of this.sessions) {
if (!session.subscribed || !session.events.has(event.type) || !matchesSelectedSource(session.sources, sourceId)) continue;
this.enqueue(session, "event", { source_id: sourceId, event: sanitizeEvent(event) }, `event:${event.id}`);
}
}
enqueue(session, type, payload, key) {
const now = Date.now();
if (session.seen.has(key)) return;
session.seen.set(key, now);
while (session.seen.size > MAX_DEDUPE) session.seen.delete(session.seen.keys().next().value);
const envelope = { type, sequence: ++this.sequence, sent_at: now, ...payload };
if (session.queue.length >= MAX_QUEUE) session.queue.shift();
session.queue.push(envelope);
this.drain(session);
}
acceptPublished(key) {
this.publishedSeen ||= new Map();
if (this.publishedSeen.has(key)) return false;
this.publishedSeen.set(key, Date.now());
while (this.publishedSeen.size > MAX_DEDUPE) this.publishedSeen.delete(this.publishedSeen.keys().next().value);
return true;
}
drain(session) {
if (session.draining) return;
session.draining = true;
try {
while (session.queue.length && session.socket.readyState === WebSocket.OPEN) {
session.socket.send(JSON.stringify(session.queue.shift()));
}
} finally { session.draining = false; }
}
send(session, type, payload) {
if (session.socket.readyState === WebSocket.OPEN) session.socket.send(JSON.stringify({ type, sequence: ++this.sequence, sent_at: Date.now(), ...payload }));
}
async close() {
this.stopChat();
this.stopEvent();
for (const session of this.sessions) session.socket.close(1001, "plugin_shutdown");
await new Promise((resolve) => this.wss.close(resolve));
}
}
function matchesSelectedSource(selected, sourceId) {
if (selected.has(sourceId)) return true;
if (sourceId.startsWith("discord:") && sourceId.endsWith(":*"))
return [...selected].some((id) => id.startsWith(sourceId.slice(0, -1)));
if (sourceId.endsWith(":current"))
return [...selected].some((id) => id.startsWith(sourceId.split(":")[0] + ":"));
return false;
}
function reject(socket, status, reason) {
const labels = { 400: "Bad Request", 401: "Unauthorized", 403: "Forbidden", 426: "Upgrade Required" };
socket.write(`HTTP/1.1 ${status} ${labels[status] || "Rejected"}\r\nConnection: close\r\nContent-Type: application/json\r\n\r\n${JSON.stringify({ error: reason })}`);
socket.destroy();
}
module.exports = { OverlayFeed, MAX_QUEUE, matchesSelectedSource };

View File

@ -0,0 +1,108 @@
"use strict";
const MAX_MESSAGE_BYTES = 64 * 1024;
const MAX_SELECTED_SOURCES = 250;
const MAX_SELECTED_EVENTS = 100;
function parseClientMessage(data) {
const body = Buffer.from(data);
if (body.length > MAX_MESSAGE_BYTES) throw coded("MESSAGE_TOO_LARGE", "Overlay control message is too large.");
let value;
try { value = JSON.parse(body.toString("utf8")); } catch { throw coded("INVALID_JSON", "Overlay control message must be valid JSON."); }
if (!value || typeof value !== "object" || Array.isArray(value)) throw coded("INVALID_MESSAGE", "Overlay control message must be an object.");
const type = String(value.type || "").trim();
if (!["subscribe", "ping", "pong"].includes(type)) throw coded("INVALID_MESSAGE", `Unsupported overlay control message '${type || "empty"}'.`);
if (type !== "subscribe") return { type };
return {
type,
sources: uniqueStrings(value.sources, MAX_SELECTED_SOURCES, 300),
events: uniqueStrings(value.events, MAX_SELECTED_EVENTS, 120)
};
}
function uniqueStrings(value, limit, maxLength) {
if (!Array.isArray(value)) return [];
return [...new Set(value.map((entry) => String(entry || "").trim().slice(0, maxLength)).filter(Boolean))].slice(0, limit);
}
function cleanText(value, max = 500) {
return String(value === undefined || value === null ? "" : value).slice(0, max);
}
function sanitizeChat(message) {
return {
id: cleanText(message.id, 256),
platform: cleanText(message.platform, 24),
text: cleanText(message.text, 2000),
timestamp: Number(message.timestamp) || Date.now(),
channel: {
id: cleanText(message.channel?.id, 256),
name: cleanText(message.channel?.name, 256),
key: cleanText(message.channel?.key, 256)
},
author: {
id: cleanText(message.author?.id, 160),
name: cleanText(message.author?.name, 160),
username: cleanText(message.author?.username, 160),
avatar: safeHttps(message.author?.avatar),
color: /^#[0-9a-f]{6}$/i.test(message.author?.color || "") ? message.author.color : null,
badges: (Array.isArray(message.author?.badges) ? message.author.badges : []).slice(0, 12).map((badge) => ({
label: cleanText(badge?.label, 40),
image: safeHttps(badge?.image)
}))
},
emotes: (Array.isArray(message.emotes) ? message.emotes : []).slice(0, 100).map((emote) => ({
start: Math.max(0, Number(emote?.start) || 0),
end: Math.max(0, Number(emote?.end) || 0),
label: cleanText(emote?.label, 100),
image: safeHttps(emote?.image)
})).filter((emote) => emote.image),
media: (Array.isArray(message.media) ? message.media : []).slice(0, 4).map((media) => ({
url: safeHttps(media?.url),
type: media?.type === "video" ? "video" : "image",
alt: cleanText(media?.alt, 160),
preview: safeHttps(media?.preview)
})).filter((media) => media.url)
};
}
function sanitizeEvent(event) {
return {
id: cleanText(event.id, 256),
type: cleanText(event.type, 120),
source: cleanText(event.source, 80),
occurred_at: Number(event.occurredAt) || Date.now(),
payload: sanitizeObject(event.payload, 0)
};
}
function sanitizeObject(value, depth) {
if (depth > 3 || value === null || value === undefined) return null;
if (Array.isArray(value)) return value.slice(0, 20).map((item) => sanitizeObject(item, depth + 1));
if (typeof value === "object") {
const output = {};
for (const [key, item] of Object.entries(value).slice(0, 50)) {
if (/token|secret|authorization|credential|oauth/i.test(key)) continue;
output[cleanText(key, 80)] = sanitizeObject(item, depth + 1);
}
return output;
}
if (typeof value === "string") return cleanText(value, 1000);
return typeof value === "number" || typeof value === "boolean" ? value : null;
}
function safeHttps(value) {
try {
const parsed = new URL(String(value || ""));
return parsed.protocol === "https:" ? parsed.toString() : null;
} catch { return null; }
}
function coded(code, message) { return Object.assign(new Error(message), { code }); }
module.exports = {
MAX_MESSAGE_BYTES,
parseClientMessage,
sanitizeChat,
sanitizeEvent
};

View File

@ -0,0 +1,86 @@
"use strict";
const { configuredChannels } = require("../../../src/services/twitch-eventsub");
const { getClient: getYouTubeClient } = require("../../../src/services/youtube");
const { getClient: getDiscordClient } = require("../../../src/services/discord");
const { twitchEventSubManager } = require("../../../src/services/twitch-eventsub");
const { listEventTypes } = require("../../../src/services/lumi-events");
const { getPlatformLiveState } = require("../../../src/services/platform-live-state");
const { getSetting } = require("../../../src/services/settings");
function listSources() {
const sources = configuredChannels().map((channel) => ({
id: `twitch:${channel}`, platform: "twitch", label: `Twitch · ${channel}`, channel_id: channel,
live: getPlatformLiveState("twitch", channel)?.live ?? null, available: true,
status: getPlatformLiveState("twitch", channel)?.live ? "live" : "configured"
}));
const youtube = getYouTubeClient();
const youtubeChannelId = youtube?.channelId || getSetting("youtube_bot_channel_id", "");
if (youtubeChannelId) sources.push({
id: `youtube:${youtubeChannelId}`, platform: "youtube", label: `YouTube · ${youtube?.channelName || youtubeChannelId}`,
channel_id: youtubeChannelId, live: youtube ? Boolean(youtube.liveChatId) : null, available: Boolean(youtube),
status: youtube ? (youtube.liveChatId ? "live" : "offline") : "not connected"
});
const discord = getDiscordClient();
for (const guild of discord?.guilds?.cache?.values?.() || []) {
for (const channel of guild.channels?.cache?.values?.() || []) {
if (!isReadableTextChannel(channel, discord)) continue;
sources.push({
id: `discord:${guild.id}:${channel.id}`, platform: "discord",
label: `Discord · ${guild.name} / #${channel.name || channel.id}`,
channel_id: channel.id, guild_id: guild.id, live: null, available: true, status: "connected"
});
}
}
return sources.sort((left, right) => left.label.localeCompare(right.label));
}
function listStatuses() {
const youtube = getYouTubeClient();
const discord = getDiscordClient();
const twitch = twitchEventSubManager.getStatus();
const missing = Object.values(twitch.capabilities || {}).filter((value) => String(value).startsWith("missing "));
return [
{ platform: "twitch", ...twitch, detail: `${twitch.detail}${missing.length ? ` Limited events: ${missing.join(", ")}.` : ""}` },
{ platform: "youtube", state: youtube ? "connected" : "unavailable", detail: youtube ? (youtube.liveChatId ? "Receiving live chat." : "Connected; no live chat is active.") : "YouTube is not configured." },
{ platform: "discord", state: discord?.isReady?.() ? "connected" : "unavailable", detail: discord?.isReady?.() ? "Discord is connected." : "Discord is not connected." }
];
}
function sourceIdForChat(message) {
if (message.platform === "twitch") return `twitch:${normalize(message.channel?.name || message.channel?.key || message.channel?.id)}`;
if (message.platform === "youtube") {
const youtube = getYouTubeClient();
return `youtube:${youtube?.channelId || message.channel?.id || normalize(message.channel?.key || message.channel?.name)}`;
}
if (message.platform === "discord") {
const [guildId, channelId] = String(message.channel?.key || "").split("/");
return `discord:${guildId || "unknown"}:${channelId || message.channel?.id || "unknown"}`;
}
return `${normalize(message.platform)}:${normalize(message.channel?.id || message.channel?.key)}`;
}
function sourceIdForEvent(event) {
const payload = event.payload || {};
if (event.type.startsWith("twitch.")) return `twitch:${normalize(payload.broadcaster_name || payload.broadcaster_id)}`;
if (event.type.startsWith("youtube.")) return `youtube:${payload.channel_id || payload.live_chat_id || "current"}`;
if (event.type.startsWith("discord.")) return `discord:${payload.guild_id || "unknown"}:${payload.channel_id || "*"}`;
return "";
}
function isReadableTextChannel(channel, discord) {
const text = typeof channel?.isTextBased === "function" ? channel.isTextBased() : ["GUILD_TEXT", 0].includes(channel?.type);
if (!text || !channel?.guild) return false;
try {
const permissions = channel.permissionsFor?.(discord.user);
if (!permissions) return true;
for (const flag of ["ViewChannel", "VIEW_CHANNEL", 1024n]) {
try { if (permissions.has?.(flag)) return true; } catch {}
}
return false;
} catch { return false; }
}
function normalize(value) { return String(value || "").trim().toLowerCase().replace(/^#/, ""); }
module.exports = { listSources, listStatuses, sourceIdForChat, sourceIdForEvent, listEventTypes, isReadableTextChannel };

View File

@ -0,0 +1,19 @@
"use strict";
const { DeviceStore } = require("../lumi_transcription/backend/companion/device_store");
const { OverlayFeed } = require("./backend/feed");
module.exports = {
id: "lumi_overlay",
init({ web, db, logger }) {
const devices = new DeviceStore(db);
const feed = new OverlayFeed({ devices, logger });
const unregister = web.addUpgradeHandler("/plugins/lumi_overlay/live", (request, socket, head) =>
feed.upgrade(request, socket, head));
logger?.info?.("Native Companion overlay feed started", {}, { event: "overlay_feed_started" });
return async () => {
unregister();
await feed.close();
};
}
};

View File

@ -0,0 +1,10 @@
{
"id": "lumi_overlay",
"name": "Lumi Overlay Feed",
"version": "0.1.1",
"description": "Authenticated live chat and event feed for the native Lumi Companion monitor overlay.",
"main": "index.js",
"channel": "stable",
"compatible_from": "0.1.0",
"rollback_safe": true
}

View File

@ -0,0 +1,136 @@
"use strict";
const assert = require("assert");
const { WebSocket } = require("ws");
const { parseClientMessage, sanitizeChat, sanitizeEvent, MAX_MESSAGE_BYTES } = require("../backend/protocol");
const { OverlayFeed, MAX_QUEUE, matchesSelectedSource } = require("../backend/feed");
const { isReadableTextChannel } = require("../backend/sources");
function run() {
verifyProtocol();
verifyCapabilityGate();
verifyFilteringAndNoBacklog();
verifyDiscordReadability();
verifyBoundedQueueAndDeduplication();
verifySanitization();
console.log("Lumi native overlay verification passed: protocol limits, server-side filtering, no backlog, reconnect-safe deduplication, bounded queues, and payload sanitization.");
}
function verifyDiscordReadability() {
const discord = { user: { id: "bot" } };
const channel = {
guild: { id: "guild" },
isTextBased: () => true,
permissionsFor: () => ({ has: (flag) => flag === 1024n })
};
assert.equal(isReadableTextChannel(channel, discord), true);
assert.equal(isReadableTextChannel({ ...channel, permissionsFor: () => ({ has: () => false }) }, discord), false);
assert.equal(isReadableTextChannel({ ...channel, isTextBased: () => false }, discord), false);
}
function verifyCapabilityGate() {
const feed = Object.create(OverlayFeed.prototype);
let response = "";
const socket = {
write(value) { response += value; },
destroy() {}
};
feed.devices = { authenticate: (_header, capability) => {
assert.equal(capability, "overlay.read.v1");
return { allowed: false, reason: "missing_credentials" };
} };
feed.upgrade({ headers: {}, socket: {} }, socket, Buffer.alloc(0));
assert.match(response, /^HTTP\/1.1 401/);
response = "";
feed.devices = { authenticate: () => ({ allowed: false, reason: "capability_revoked" }) };
feed.upgrade({ headers: { authorization: "redacted" }, socket: {} }, socket, Buffer.alloc(0));
assert.match(response, /^HTTP\/1.1 403/);
}
function verifyProtocol() {
assert.deepEqual(parseClientMessage(JSON.stringify({
type: "subscribe",
sources: ["twitch:lumi", "twitch:lumi", "discord:guild:channel"],
events: ["twitch.follow"]
})), {
type: "subscribe",
sources: ["twitch:lumi", "discord:guild:channel"],
events: ["twitch.follow"]
});
assert.throws(() => parseClientMessage(JSON.stringify({ type: "history" })), /Unsupported/);
assert.throws(() => parseClientMessage(Buffer.alloc(MAX_MESSAGE_BYTES + 1)), /too large/);
}
function verifyFilteringAndNoBacklog() {
const feed = Object.create(OverlayFeed.prototype);
feed.sessions = new Set();
feed.sequence = 0;
feed.publishedSeen = new Map();
feed.publishChat({ id: "before", platform: "twitch", channel: { name: "lumi" } });
const sent = [];
const session = sessionFor(sent, ["twitch:lumi"]);
feed.sessions.add(session);
assert.equal(sent.length, 0, "opening/subscribing must not replay a pre-connection message");
feed.publishChat({
id: "selected", platform: "twitch", text: "hello", timestamp: 1,
channel: { id: "1", name: "lumi", key: "lumi" },
author: { id: "viewer", name: "Viewer", badges: [] }, emotes: []
});
feed.publishChat({
id: "selected", platform: "twitch", text: "duplicate", timestamp: 3,
channel: { id: "1", name: "lumi", key: "lumi" },
author: { id: "viewer", name: "Viewer", badges: [] }, emotes: []
});
feed.publishChat({
id: "other", platform: "twitch", text: "hidden", timestamp: 2,
channel: { id: "2", name: "other", key: "other" },
author: { id: "viewer", name: "Viewer", badges: [] }, emotes: []
});
assert.equal(sent.length, 1, "provider duplicates must remain suppressed across feed sessions/reconnects");
assert.equal(JSON.parse(sent[0]).message.id, "selected");
assert.equal(matchesSelectedSource(new Set(["discord:guild:channel"]), "discord:guild:*"), true);
}
function verifyBoundedQueueAndDeduplication() {
const feed = Object.create(OverlayFeed.prototype);
feed.sequence = 0;
const session = sessionFor([], ["twitch:lumi"], WebSocket.CLOSED);
for (let index = 0; index < MAX_QUEUE + 40; index += 1)
feed.enqueue(session, "chat", { index }, `message:${index}`);
assert.equal(session.queue.length, MAX_QUEUE);
assert.equal(session.queue[0].index, 40, "overflow must deterministically drop the oldest queued item");
feed.enqueue(session, "chat", { index: 999 }, `message:${MAX_QUEUE + 39}`);
assert.equal(session.queue.length, MAX_QUEUE, "a duplicate event must not enter the queue twice");
}
function verifySanitization() {
const chat = sanitizeChat({
id: "id", platform: "twitch", text: "<script>plain text</script>",
channel: {}, author: { id: "user", name: "Viewer", avatar: "http://insecure.test/avatar.png" },
emotes: [{ start: 0, end: 1, image: "https://cdn.test/emote.png" }]
});
assert.equal(chat.text, "<script>plain text</script>", "content remains plain data and is not interpreted server-side");
assert.equal(chat.author.avatar, null);
assert.equal(chat.emotes[0].image, "https://cdn.test/emote.png");
const discord = sanitizeChat({
id: "discord-id", platform: "discord", text: "<a:wave:1>", channel: {},
author: { id: "discord-user", name: "Discord viewer", badges: [] },
emotes: [{ start: 0, end: 9, label: ":wave:", image: "https://cdn.test/wave.gif" }],
media: [{ url: "https://cdn.test/reaction.gif", type: "image", alt: "Reaction" }]
});
assert.equal(discord.emotes[0].image, "https://cdn.test/wave.gif", "Discord emotes must reach the Companion feed");
assert.equal(discord.media[0].url, "https://cdn.test/reaction.gif", "Discord GIFs must reach the Companion feed");
const event = sanitizeEvent({ id: "event", type: "twitch.follow", payload: { user_name: "Viewer", access_token: "secret" } });
assert.equal(event.payload.access_token, undefined);
assert.equal(event.payload.user_name, "Viewer");
}
function sessionFor(sent, sources, state = WebSocket.OPEN) {
return {
socket: { readyState: state, send(value) { sent.push(value); } },
device: {}, sources: new Set(sources), events: new Set(), subscribed: true,
queue: [], draining: false, seen: new Map(), lastPong: Date.now()
};
}
run();

View File

@ -1,5 +1,61 @@
# Lumi Transcription changelog
## 0.2.8
- Release Companion 0.2.8 with reusable click-to-capture hotkeys, independently switchable Windows media sources, and direct drag/resize editing through the actual native Lumi Overlay renderer.
- Preserve paired identity, transcription settings and behavior, OBS Bridge 0.2.5, Song Overlay settings, native overlay settings, and automatic update compatibility.
## 0.2.7
- Release Companion 0.2.7 with animated native overlay emotes, Discord GIF
media, and Twitch badge artwork.
- Retain the existing OBS browser overlay renderer, transcription behavior,
OBS Bridge 0.2.5, paired identity, and automatic update compatibility.
## 0.2.6
- Release Companion 0.2.6 with the native Lumi Overlay and shell-owned authenticated live feed.
- Preserve unsaved overlay settings through periodic catalog/status refreshes and improve percentage, visibility, layout, monitor, source, and event presentation.
- Keep Song Overlay and Lumi Overlay as direct single-page navigation entries while retaining nested Transcription pages.
- Retain OBS Bridge 0.2.5, current transcription behavior, paired identity, settings, and automatic update compatibility.
## 0.2.5
- Add encrypted Domeneshop DNS-01 automation for RTMPS certificate issuance and renewal when the production HTTPS proxy reserves the HTTP challenge route.
- Verify credentials in the Stream Testing WebUI, wait against authoritative DNS, and clean up temporary TXT records after success or failure.
- Retain HTTP-01 where it is forwarded correctly and provide actionable proxy-specific failure guidance otherwise.
- Release matching Companion 0.2.5 while retaining the current OBS Bridge 0.2.5.
## 0.2.4
- Reveal private-test captions progressively, wrap them across readable lines, and age stale caption words out without waiting for a complete sentence.
- Add a Companion transcription toggle, silent connection recovery, consistent collapsed plugin navigation, and direct access to the active Lumi Stream Testing viewer.
- Support Lumi-managed RTMPS ingest certificates in production while retaining explicit localhost and private-network development transport.
- Release Companion 0.2.4 with OBS Bridge 0.2.5 through the normal verified installer and update path.
## 0.2.3
- Added the MediaMTX private-test destination contract and OBS Bridge 0.2.4.
- Isolated private tests in a dedicated libobs output so Twitch OAuth cannot replace the temporary destination, while preserving exact crash-safe service restoration.
- Added process-aware failed-start recovery and a bounded retry when OBS is still stopping the private output.
## 0.2.2
- Fixed the managed OBS Bridge's streaming-service ownership so starting a private Stream Test no longer releases OBS's current service and crashes OBS.
- Require the corrected bridge version before Stream Testing and reuse the existing Update OBS Bridge action to install it safely while OBS is closed.
## 0.2.1
- Surface Stream Testing startup failures immediately instead of leaving the Companion action waiting with a stale status.
- Detect the OBS Bridge version actually loaded by OBS and show Update OBS Bridge actions on Overview and Connection & device when it differs from the bundled integration.
- Prevent private output redirection until an outdated or damaged OBS integration is updated or repaired.
## 0.2.0
- Added authenticated private stream-test session control over the existing Companion transport, including reused live caption cues and OBS output diagnostics.
- Added crash-safe DPAPI recovery for exact OBS streaming-service restoration, with automatic startup/exit/server-loss repair and no local speech inference.
- Added repeatable single-flight Companion update checks with success, current, failure, cancellation, and rapid-click regression coverage.
## 0.1.0
- Added server-hosted whisper.cpp transcription with bounded Companion audio transport and no local speech inference.

View File

@ -1,6 +1,12 @@
const crypto = require("crypto");
const DEFAULT_CAPABILITIES = Object.freeze(["transcription.capture.v1", "transcription.settings.v1", "obs.caption.native.v1"]);
const DEFAULT_CAPABILITIES = Object.freeze([
"transcription.capture.v1",
"transcription.settings.v1",
"obs.caption.native.v1",
"overlay.read.v1"
]);
const OVERLAY_CAPABILITY_MIGRATION = "overlay-read-v1";
class DeviceStore {
constructor(db, options = {}) {
@ -24,8 +30,26 @@ class DeviceStore {
pairing_host TEXT
);
CREATE INDEX IF NOT EXISTS transcription_devices_user_idx ON transcription_devices(lumi_user_id);
CREATE TABLE IF NOT EXISTS companion_capability_migrations (
id TEXT PRIMARY KEY, applied_at INTEGER NOT NULL
);
`);
ensureColumn(this.db, "transcription_devices", "pairing_host", "TEXT");
this.migrateOverlayCapabilityOnce();
}
migrateOverlayCapabilityOnce() {
if (this.db.prepare("SELECT 1 FROM companion_capability_migrations WHERE id = ?").get(OVERLAY_CAPABILITY_MIGRATION)) return;
const migrate = this.db.transaction(() => {
const rows = this.db.prepare("SELECT id, capabilities_json FROM transcription_devices WHERE revoked_at IS NULL").all();
const update = this.db.prepare("UPDATE transcription_devices SET capabilities_json = ? WHERE id = ? AND revoked_at IS NULL");
for (const row of rows) {
const capabilities = parseArray(row.capabilities_json);
if (!capabilities.includes("overlay.read.v1")) update.run(JSON.stringify([...capabilities, "overlay.read.v1"]), row.id);
}
this.db.prepare("INSERT OR IGNORE INTO companion_capability_migrations (id, applied_at) VALUES (?, ?)").run(OVERLAY_CAPABILITY_MIGRATION, this.now());
});
migrate();
}
issuePairing({ userId, host, ttlMs = 15 * 60 * 1000 }) {

View File

@ -1,6 +1,10 @@
const { WebSocketServer, WebSocket } = require("ws");
const { MAX_JSON_BYTES, MAX_AUDIO_PAYLOAD_BYTES, AUDIO_HEADER_BYTES, parseEnvelope, validateHello, parseAudioFrame, envelope } = require("./protocol");
const { insecureDeviceAllowed } = require("./device_store");
const {
isSecureRequest,
requestOrigin
} = require("../../../../src/services/proxy-security");
const MAX_CONTROL_MESSAGES_PER_SECOND = 120;
const MAX_AUDIO_MESSAGES_PER_SECOND = 200;
@ -15,15 +19,18 @@ class CompanionGateway {
this.wss.on("connection", (socket, request, device) => this.connection(socket, request, device));
}
upgrade(request, socket, head) {
const forwardedProto = String(request.headers["x-forwarded-proto"] || "").split(",")[0].trim();
const proxyIsLocal = ["127.0.0.1", "::1", "::ffff:127.0.0.1"].includes(request.socket.remoteAddress);
const secure = Boolean(request.socket.encrypted) || (proxyIsLocal && forwardedProto === "https");
const secure = isSecureRequest(request);
const auth = this.devices.authenticate(request.headers.authorization, "transcription.capture.v1");
if (!auth.allowed) return reject(socket, auth.reason === "capability_revoked" ? 403 : 401, auth.reason);
const requestOrigin = `http://${request.headers.host || "invalid"}`;
if (!secure && !insecureDeviceAllowed(auth.device, requestOrigin, request.socket.remoteAddress)) return reject(socket, 426, "tls_required");
const origin = request.headers.origin;
if (origin && !sameHostOrigin(origin, request.headers.host)) return reject(socket, 403, "origin_rejected");
let requestOriginValue;
try {
requestOriginValue = requestOrigin(request);
} catch {
return reject(socket, 400, "invalid_host");
}
if (!secure && !insecureDeviceAllowed(auth.device, requestOriginValue, request.socket.remoteAddress)) return reject(socket, 426, "tls_required");
const browserOrigin = request.headers.origin;
if (browserOrigin && !sameHostOrigin(browserOrigin, request.headers.host)) return reject(socket, 403, "origin_rejected");
this.wss.handleUpgrade(request, socket, head, (ws) => this.wss.emit("connection", ws, request, auth.device));
}
connection(socket, _request, device) {
@ -95,7 +102,7 @@ class CompanionGateway {
this.log.append({ kind: "connection", state: "authenticated", device_id: device.id, session_id: session.id });
return;
}
await this.structured(session, message, send, () => { lastPong = Date.now(); });
await this.structured(session, device, message, send, () => { lastPong = Date.now(); });
} catch (error) {
send("error", { code: error.code || "INVALID_MESSAGE", message: error.message, recoverable: !["INCOMPATIBLE_VERSION", "HELLO_REQUIRED"].includes(error.code) });
this.log.append({ kind: "protocol_error", device_id: device.id, session_id: session?.id, code: error.code || "INVALID_MESSAGE", message: error.message });
@ -106,11 +113,17 @@ class CompanionGateway {
socket.on("close", () => {
clearTimeout(helloTimer); clearInterval(heartbeat);
if (session) this.sessions.disconnect(session.id);
if (session) {
void global.lumiFrameworks?.streamTesting?.stop?.({
device_id: device.id,
reason: "Companion disconnected; the private receiver was stopped."
}).catch(() => {});
}
this.log.append({ kind: "connection", state: "closed", device_id: device.id, session_id: session?.id });
});
socket.on("error", (error) => this.log.append({ kind: "connection", state: "error", device_id: device.id, session_id: session?.id, message: error.message }));
}
async structured(session, message, send, pong) {
async structured(session, device, message, send, pong) {
switch (message.type) {
case "ping":
pong();
@ -132,6 +145,37 @@ class CompanionGateway {
case "readiness": send("status", { kind: "readiness", ...(await this.sessions.readiness()), server_plugin_version: require("../../plugin.json").version }); break;
case "start": send("status", { kind: "session", ...(await this.sessions.start(session.id, message.payload || {})) }); break;
case "stop": send("status", { kind: "session", ...(await this.sessions.stop(session.id, cleanReason(message.payload?.reason))) }); break;
case "stream_test_create": {
const service = global.lumiFrameworks?.streamTesting;
if (!service) throw coded("STREAM_TEST_UNAVAILABLE", "Stream testing is not available on this Lumi host.");
const created = await service.create(device, message.payload || {}, send);
send("stream_test_session", created);
break;
}
case "stream_test_obs_metrics": {
const service = global.lumiFrameworks?.streamTesting;
if (!service) throw coded("STREAM_TEST_UNAVAILABLE", "Stream testing is not available on this Lumi host.");
service.updateObs(device.id, message.payload || {});
break;
}
case "stream_test_caption_status": {
const service = global.lumiFrameworks?.streamTesting;
if (!service) throw coded("STREAM_TEST_UNAVAILABLE", "Stream testing is not available on this Lumi host.");
service.updateCaptionStatus(device.id, message.payload || {});
break;
}
case "stream_test_caption": {
const service = global.lumiFrameworks?.streamTesting;
if (!service) throw coded("STREAM_TEST_UNAVAILABLE", "Stream testing is not available on this Lumi host.");
service.addCaption(device.id, message.payload || {});
break;
}
case "stream_test_stop": {
const service = global.lumiFrameworks?.streamTesting;
if (!service) throw coded("STREAM_TEST_UNAVAILABLE", "Stream testing is not available on this Lumi host.");
await service.stop({ device_id: device.id, session_id: message.payload?.session_id, reason: cleanStreamTestReason(message.payload?.reason) });
break;
}
case "ack": break;
default: throw coded("UNEXPECTED_MESSAGE", `Message ${message.type} is not valid after the handshake.`);
}
@ -147,5 +191,6 @@ function closeWith(socket, code, reason) { if (socket.readyState === WebSocket.O
function sameHostOrigin(origin, host) { try { return new URL(origin).host === host; } catch { return false; } }
function coded(code, message) { return Object.assign(new Error(message), { code }); }
function cleanReason(value) { return ["requested", "test_complete", "benchmark_complete", "silence_timeout", "disconnect"].includes(String(value)) ? String(value) : "requested"; }
function cleanStreamTestReason(value) { return ["requested", "companion_exit", "obs_stopped", "startup_recovery", "server_ended"].includes(String(value)) ? String(value) : "requested"; }
module.exports = { CompanionGateway, sameHostOrigin, MAX_CONTROL_MESSAGES_PER_SECOND, MAX_AUDIO_MESSAGES_PER_SECOND, MAX_SOURCE_MESSAGES_PER_SECOND };

View File

@ -5,7 +5,8 @@ const AUDIO_MAGIC = Buffer.from("LACP");
const AUDIO_HEADER_BYTES = 64;
const MAX_JSON_BYTES = 64 * 1024;
const MAX_AUDIO_PAYLOAD_BYTES = 6400;
const CLIENT_TYPES = new Set(["hello", "ping", "source_update", "obs_state", "readiness", "start", "stop", "ack"]);
const CLIENT_TYPES = new Set(["hello", "ping", "source_update", "obs_state", "readiness", "start", "stop", "ack",
"stream_test_create", "stream_test_obs_metrics", "stream_test_caption_status", "stream_test_caption", "stream_test_stop"]);
function parseEnvelope(input) {
const bytes = Buffer.isBuffer(input) ? input : Buffer.from(String(input || ""), "utf8");

View File

@ -42,14 +42,27 @@ class JsonlDiagnosticLog {
function sanitize(value, includeCaptionText) {
if (Buffer.isBuffer(value)) return "[binary omitted]";
if (Array.isArray(value)) return value.map((entry) => sanitize(entry, includeCaptionText));
if (typeof value === "string") return sanitizeText(value);
if (!value || typeof value !== "object") return value;
const result = {};
for (const [key, child] of Object.entries(value)) {
if (/audio|pcm|credential|secret|token/i.test(key)) result[key] = "[redacted]";
if (isSensitiveKey(key)) result[key] = "[redacted]";
else if (!includeCaptionText && /(?:stable|uncertain|caption)_text/i.test(key)) result[key] = "[caption text disabled]";
else result[key] = sanitize(child, includeCaptionText);
}
return result;
}
module.exports = { JsonlDiagnosticLog, sanitize };
function isSensitiveKey(key) {
return /(?:audio|pcm|credential|(?:^|[_-])(?:authorization|cookie|password|passwd|secret|signature|token|api[_-]?key|private[_-]?key)(?:$|[_-]))/i.test(String(key));
}
function sanitizeText(value) {
return String(value || "")
.replace(/\b(Bearer|Basic|LumiDevice)\s+[A-Za-z0-9._~+/=-]+/gi, "$1 [redacted]")
.replace(/\b((?:set-)?cookie\s*:\s*)[^\r\n]+/gi, "$1[redacted]")
.replace(/([?&](?:token|key|secret|password|authorization)=)[^&#\s]+/gi, "$1[redacted]")
.replace(/\b((?:api[_-]?key|access[_-]?token|refresh[_-]?token|client[_-]?secret|password|passwd|secret|signature|authorization)\s*[:=]\s*)[^\s,;}]+/gi, "$1[redacted]");
}
module.exports = { JsonlDiagnosticLog, sanitize, sanitizeText };

View File

@ -1,90 +1 @@
const crypto = require("crypto");
const fs = require("fs");
const path = require("path");
const AdmZip = require("adm-zip");
class ArtifactManager {
constructor(root, options = {}) { this.root = root; this.fetch = options.fetch || global.fetch; fs.mkdirSync(root, { recursive: true }); }
status(entry) {
const target = path.join(this.root, entry.filename || entry.id);
if (!fs.existsSync(target)) return { installed: false, valid: false, path: target };
return { installed: true, valid: sha256File(target) === entry.sha256, path: target, bytes: fs.statSync(target).size };
}
async download(entry, options = {}) {
validateManifestEntry(entry);
if (options.confirmed !== true) throw new Error("Download requires explicit setup confirmation.");
const filename = entry.filename || path.basename(new URL(entry.url).pathname);
const target = path.join(this.root, filename);
const partial = `${target}.${process.pid}.partial`;
fs.rmSync(partial, { force: true });
try {
const response = await this.fetch(entry.url, { redirect: "follow" });
if (!response.ok || !response.body) throw new Error(`Download failed with HTTP ${response.status}.`);
const handle = fs.createWriteStream(partial, { flags: "wx", mode: 0o600 });
for await (const chunk of response.body) {
if (!handle.write(chunk)) await new Promise((resolve) => handle.once("drain", resolve));
}
await new Promise((resolve, reject) => handle.end((error) => error ? reject(error) : resolve()));
if (Number.isSafeInteger(entry.bytes) && fs.statSync(partial).size !== entry.bytes) throw new Error(`Downloaded size mismatch for ${entry.id}.`);
const actual = sha256File(partial);
if (actual !== entry.sha256) throw new Error(`Checksum mismatch for ${entry.id}.`);
fs.renameSync(partial, target);
return this.status({ ...entry, filename });
} finally { fs.rmSync(partial, { force: true }); }
}
installZip(entry, archivePath) {
validateManifestEntry(entry);
if (sha256File(archivePath) !== entry.sha256) throw new Error(`Checksum mismatch for ${entry.id}.`);
const target = path.join(this.root, entry.id);
const staged = `${target}.${process.pid}.staged`;
fs.rmSync(staged, { recursive: true, force: true });
fs.mkdirSync(staged, { recursive: true });
const zip = new AdmZip(archivePath);
for (const item of zip.getEntries()) {
const portable = String(item.entryName).replace(/\\/g, "/");
const relative = path.posix.normalize(portable).replace(/^\/+/, "");
if (!relative || relative === ".." || relative.startsWith("../") || /^[A-Za-z]:/.test(relative)) throw new Error("Runtime archive contains an unsafe path.");
}
zip.extractAllTo(staged, true);
for (const expected of entry.expected_paths || []) if (!findBasename(staged, expected)) throw new Error(`Runtime is missing ${expected}.`);
replaceDirectory(staged, target);
return { installed: true, path: target, backend: entry.backend, version: entry.id };
}
}
function validateManifestEntry(entry) { if (!entry?.id || !/^https:\/\//.test(entry.url || "") || !/^[a-f0-9]{64}$/.test(entry.sha256 || "")) throw new Error("Artifact manifest entry is invalid."); }
function sha256File(target) { const hash = crypto.createHash("sha256"); const fd = fs.openSync(target, "r"); const buffer = Buffer.alloc(1024 * 1024); try { let read; while ((read = fs.readSync(fd, buffer, 0, buffer.length, null)) > 0) hash.update(buffer.subarray(0, read)); } finally { fs.closeSync(fd); } return hash.digest("hex"); }
function findBasename(root, basename) { const pending = [root]; while (pending.length) { const current = pending.pop(); for (const entry of fs.readdirSync(current, { withFileTypes: true })) { const target = path.join(current, entry.name); if (entry.isDirectory()) pending.push(target); else if (entry.name === basename) return target; } } return null; }
function replaceDirectory(staged, target) {
const backup = `${target}.${process.pid}.backup`;
fs.rmSync(backup, { recursive: true, force: true });
const hadTarget = fs.existsSync(target);
let backupDisposable = false;
try {
if (hadTarget) moveDirectory(target, backup);
moveDirectory(staged, target);
backupDisposable = true;
fs.rmSync(backup, { recursive: true, force: true });
} catch (error) {
fs.rmSync(target, { recursive: true, force: true });
if (hadTarget && fs.existsSync(backup)) {
moveDirectory(backup, target);
backupDisposable = true;
} else if (!hadTarget) backupDisposable = true;
throw error;
} finally {
fs.rmSync(staged, { recursive: true, force: true });
if (backupDisposable) fs.rmSync(backup, { recursive: true, force: true });
}
}
function moveDirectory(source, target) {
try { fs.renameSync(source, target); }
catch (error) {
if (!["EPERM", "EACCES", "EXDEV"].includes(error.code)) throw error;
fs.mkdirSync(target, { recursive: true });
fs.cpSync(source, target, { recursive: true, force: true });
fs.rmSync(source, { recursive: true, force: true });
}
}
module.exports = { ArtifactManager, sha256File, validateManifestEntry };
module.exports = require("../../../../src/services/artifact-manager");

View File

@ -76,7 +76,13 @@ class SessionCoordinator {
async start(sessionId, options = {}) {
const session = this.require(sessionId);
if (session.state !== "idle") throw Object.assign(new Error("Another transcription activity is still finishing. Try again in a moment."), { code: "SESSION_BUSY" });
const mode = options.mode === "test" ? "test" : options.mode === "benchmark" ? "benchmark" : "live";
const mode = options.mode === "test"
? "test"
: options.mode === "benchmark"
? "benchmark"
: options.mode === "stream_test"
? "stream_test"
: "live";
if (mode === "live" && !session.obs.streaming) throw Object.assign(new Error("Live transcription requires an active OBS stream."), { code: "OBS_NOT_STREAMING" });
const tracks = Array.from(session.tracks.values()).filter((track) => track.enabled && !track.source_missing);
if (!tracks.length) throw Object.assign(new Error("Select an available OBS audio source first."), { code: "NO_TRACKS" });

View File

@ -1,17 +1,17 @@
{
"schema_version": 1,
"version": "0.1.0",
"version": "0.2.8",
"signed": false,
"release_notes": "First stable Companion release with shared plugin authentication, server-hosted transcription, Song Overlay, matching Windows branding, and a steady non-flickering tray status.",
"release_notes": "Adds click-to-capture hotkeys, independent Windows media-source discovery, and direct drag/resize editing through the actual native Lumi Overlay renderer while retaining paired identity, transcription, Stream Testing, silent reconnect, and OBS Bridge 0.2.5.",
"installer": {
"id": "windows-x64-installer",
"platform": "win32",
"architecture": "x64",
"label": "Windows x64 per-user installer",
"filename": "Lumi.Companion-Setup.exe",
"url": "https://git.rolfsvaag.no/Rolfsvaag_Datateknikk/Lumi/releases/download/companion-v0.1.0/Lumi.Companion-Setup.exe",
"sha256": "71ee091971a8e246d9b991a16fc28bd73495351c82b44839ae34ca36fb5784ff",
"bytes": 51849287
"url": "https://git.rolfsvaag.no/Rolfsvaag_Datateknikk/Lumi/releases/download/companion-v0.2.8/Lumi.Companion-Setup.exe",
"sha256": "15d632160d29b29773e2ec692ef1061dc6d662a4a80a128ea1674ee97bdeef99",
"bytes": 51926217
},
"artifacts": [
{
@ -20,9 +20,9 @@
"architecture": "x64",
"label": "Windows x64 self-contained",
"filename": "Lumi.Companion-win-x64.zip",
"url": "https://git.rolfsvaag.no/Rolfsvaag_Datateknikk/Lumi/releases/download/companion-v0.1.0/Lumi.Companion-win-x64.zip",
"sha256": "1194475b2a6acdeb6ef9429726cb5de31a5c93f1dc53a3faa29bb54eaef0368a",
"bytes": 66731197,
"url": "https://git.rolfsvaag.no/Rolfsvaag_Datateknikk/Lumi/releases/download/companion-v0.2.8/Lumi.Companion-win-x64.zip",
"sha256": "df94e2cbccb54a8f002940f25099572fb92fc3448da2ec463f85446df55fb1a3",
"bytes": 66863322,
"entrypoint": "Lumi.Companion.App.exe"
}
]

View File

@ -2,6 +2,10 @@ const express = require("express");
const ejs = require("ejs");
const fs = require("fs");
const path = require("path");
const {
isSecureRequest,
requestOrigin
} = require("../../src/services/proxy-security");
const { DeviceStore } = require("./backend/companion/device_store");
const { insecureDeviceAllowed, sameLocalhostOrigin } = require("./backend/companion/device_store");
const { CompanionGateway } = require("./backend/companion/gateway");
@ -98,25 +102,62 @@ module.exports = {
runtime: runtimeManifest
}));
router.post("/api/pairing-package", requireAdmin, (req, res) => {
let host = "";
try {
const host = requestHost(req);
host = requestHost(req);
diagnosticLog.append?.({ kind: "companion_pairing", action: "request_start", scheme: host.startsWith("https") ? "https" : "http", host });
if (!host.startsWith("https")) {
logger.warn?.(
"Companion pairing package requested over HTTP — check X-Forwarded-Proto proxy header",
{ plugin: PLUGIN_ID, host },
{ event: "companion_http_pairing_request" }
);
}
const pairing = devices.issuePairing({ userId: req.session.user.id, host });
diagnosticLog.append?.({ kind: "companion_pairing", action: "pairing_issued", pairing_id: pairing.pairing_id, scheme: host.startsWith("https") ? "https" : "http" });
const bootstrap = { format: "lumi-companion-bootstrap-v1", ...pairing, exchange_url: `${host}/plugins/${PLUGIN_ID}/api/pair`, ...companionOperatorMetadata(host) };
res.set("Cache-Control", "no-store");
res.attachment(`lumi-companion-${pairing.pairing_id}.lumi-pairing.json`);
res.send(`${JSON.stringify(bootstrap, null, 2)}\n`);
} catch (error) { res.status(400).json({ ok: false, error: error.message }); }
} catch (error) {
diagnosticLog.append?.({ kind: "companion_pairing", action: "pairing_failed", error: error.message, scheme: host ? (host.startsWith("https") ? "https" : "http") : "invalid" });
logger.error?.(
"Failed to generate companion pairing package",
{ plugin: PLUGIN_ID, error: error.message },
{ event: "companion_pairing_failed" }
);
res.status(400).json({ ok: false, error: error.message });
}
});
router.post("/api/companion/download", requireAdmin, async (req, res) => {
let host = "";
try {
const host = requestHost(req);
host = requestHost(req);
diagnosticLog.append?.({ kind: "companion_download", action: "request_start", scheme: host.startsWith("https") ? "https" : "http", host });
if (!host.startsWith("https")) {
logger.warn?.(
"Companion download requested over HTTP — check X-Forwarded-Proto proxy header",
{ plugin: PLUGIN_ID, host },
{ event: "companion_http_download_request" }
);
}
const pairing = devices.issuePairing({ userId: req.session.user.id, host });
diagnosticLog.append?.({ kind: "companion_download", action: "pairing_issued_for_download", pairing_id: pairing.pairing_id });
const bootstrap = { format: "lumi-companion-bootstrap-v1", ...pairing, exchange_url: `${host}/plugins/${PLUGIN_ID}/api/pair`, ...companionOperatorMetadata(host) };
const bundle = await companionPackages.build({ ...pairing, bootstrap });
diagnosticLog.append?.({ kind: "companion_download", action: "bundle_built", filename: bundle.filename, bytes: bundle.buffer?.length || 0 });
res.set("Cache-Control", "no-store");
res.attachment(bundle.filename);
res.send(bundle.buffer);
} catch (error) { res.status(503).json({ ok: false, error: error.message }); }
} catch (error) {
diagnosticLog.append?.({ kind: "companion_download", action: "download_failed", error: error.message });
logger.error?.(
"Failed to build Companion download bundle",
{ plugin: PLUGIN_ID, error: error.message },
{ event: "companion_download_failed" }
);
res.status(503).json({ ok: false, error: error.message });
}
});
const companionUpdateHandler = async (req, res) => {
try {
@ -342,7 +383,7 @@ function companionOperatorMetadata(host) {
};
}
function requireDeviceAccess(devices) { return (req, res, next) => { const auth = devices.authenticate(req.headers.authorization); if (!auth.allowed) return res.status(401).json({ error: "A paired device credential is required." }); if (!req.secure && !insecureDeviceAllowed(auth.device, requestHost(req), req.socket.remoteAddress)) return res.status(426).json({ error: "Companion requests require HTTPS unless this device was paired from the same localhost origin." }); req.lumiDevice = auth.device; next(); }; }
function requireDeviceAccess(devices) { return (req, res, next) => { const auth = devices.authenticate(req.headers.authorization); if (!auth.allowed) return res.status(401).json({ error: "A paired device credential is required." }); if (!isSecureRequest(req) && !insecureDeviceAllowed(auth.device, requestHost(req), req.socket.remoteAddress)) return res.status(426).json({ error: "Companion requests require HTTPS unless this device was paired from the same localhost origin." }); req.lumiDevice = auth.device; next(); }; }
function compareVersions(left, right) {
const parse = (value) => {
const [version] = String(value || "0").split("+", 1);
@ -405,6 +446,6 @@ async function renderLumiPage(locals) {
]);
return `${top}${body}${bottom}`;
}
function requireSettingsAccess(devices) { return (req, res, next) => { if (req.session?.user?.isAdmin) return next(); const auth = devices.authenticate(req.headers.authorization, "transcription.settings.v1"); if (!auth.allowed) return res.status(401).json({ error: "A paired device credential is required." }); if (!req.secure && !insecureDeviceAllowed(auth.device, requestHost(req), req.socket.remoteAddress)) return res.status(426).json({ error: "Device settings synchronization requires HTTPS unless this device was paired from the same localhost origin." }); req.lumiDevice = auth.device; next(); }; }
function requestHost(req) { return `${req.protocol === "https" ? "https" : "http"}://${req.get("host")}`; }
function requirePairingTransport(devices) { return (req, res, next) => { if (req.secure) return next(); const local = ["127.0.0.1", "::1", "::ffff:127.0.0.1"].includes(req.socket.remoteAddress); if (local && devices.pairingAllowsHttp(req.body?.token, requestHost(req))) return next(); return res.status(426).json({ error: "Lumi Companion pairing requires HTTPS unless the package was generated from this exact localhost URL." }); }; }
function requireSettingsAccess(devices) { return (req, res, next) => { if (req.session?.user?.isAdmin) return next(); const auth = devices.authenticate(req.headers.authorization, "transcription.settings.v1"); if (!auth.allowed) return res.status(401).json({ error: "A paired device credential is required." }); if (!isSecureRequest(req) && !insecureDeviceAllowed(auth.device, requestHost(req), req.socket.remoteAddress)) return res.status(426).json({ error: "Device settings synchronization requires HTTPS unless this device was paired from the same localhost origin." }); req.lumiDevice = auth.device; next(); }; }
function requestHost(req) { return requestOrigin(req); }
function requirePairingTransport(devices) { return (req, res, next) => { if (isSecureRequest(req)) return next(); const local = ["127.0.0.1", "::1", "::ffff:127.0.0.1"].includes(req.socket.remoteAddress); if (local && devices.pairingAllowsHttp(req.body?.token, requestHost(req))) return next(); return res.status(426).json({ error: "Lumi Companion pairing requires HTTPS unless the package was generated from this exact localhost URL." }); }; }

View File

@ -1,7 +1,7 @@
{
"id": "lumi_transcription",
"name": "Lumi Transcription",
"version": "0.1.0",
"version": "0.2.8",
"description": "Server-hosted whisper.cpp transcription for Lumi Companion and OBS closed captions.",
"main": "index.js",
"channel": "stable",

View File

@ -24,6 +24,10 @@ const { CaptionStabilizer, LatestCaptionGate } = require("../backend/transcripti
const { WhisperWorkerSupervisor, WhisperCppServerProvider } = require("../backend/transcription/provider");
const { BenchmarkStore, metricStats } = require("../backend/tests/benchmark_store");
const plugin = require("../index");
const {
isSecureRequest,
requestOrigin
} = require("../../../src/services/proxy-security");
const { createWebUpgradeRegistry } = require("../../../src/services/web-upgrades");
async function run() {
@ -31,7 +35,9 @@ async function run() {
try {
verifyProtocol();
verifyPairingAndRevocation();
verifyOverlayCapabilityMigration();
verifyLocalhostTransportPolicy();
verifyProxiedHttpsTransport();
verifyCompanionVersionOrdering();
verifyWorkerResolution(temp);
verifyRevisions();
@ -53,6 +59,43 @@ async function run() {
} finally { fs.rmSync(temp, { recursive: true, force: true }); }
}
function verifyProxiedHttpsTransport() {
const request = {
protocol: "http",
secure: false,
socket: { remoteAddress: "::ffff:172.20.0.3" },
get(name) {
if (name === "x-forwarded-proto") return "https";
if (name === "host") return "lumi.example.test";
return "";
}
};
const origin = requestOrigin(request);
assert.equal(origin, "https://lumi.example.test");
assert.equal(isSecureRequest(request), true);
assert.doesNotThrow(() => normalizeHost(origin), "a Companion package generated behind the production HTTPS proxy must retain its HTTPS origin");
const devices = new DeviceStore(new Database(":memory:"));
const pairing = devices.issuePairing({ userId: "admin", host: origin });
assert.equal(pairing.host, "https://lumi.example.test", "the issued package must contain the public HTTPS origin");
assert.throws(
() => normalizeHost(requestOrigin({ ...request, socket: { remoteAddress: "203.0.113.20" } })),
/HTTPS/,
"an untrusted client cannot turn an HTTP request into an HTTPS pairing package with forwarding headers"
);
assert.throws(
() => requestOrigin({
...request,
get(name) {
if (name === "x-forwarded-proto") return "https";
if (name === "host") return "lumi.example.test/forged";
return "";
}
}),
/host is invalid/,
"pairing origins must reject malformed Host headers instead of silently normalizing them"
);
}
function verifyAdminDeviceRevocationUx() {
const view = fs.readFileSync(path.join(__dirname, "../views/settings.ejs"), "utf8");
const client = fs.readFileSync(path.join(__dirname, "../public/transcription.js"), "utf8");
@ -102,6 +145,37 @@ function verifyPairingAndRevocation() {
db.close();
}
function verifyOverlayCapabilityMigration() {
const db = new Database(":memory:");
db.exec(`
CREATE TABLE transcription_pairing_tokens (
id TEXT PRIMARY KEY, token_hash TEXT NOT NULL UNIQUE, lumi_user_id TEXT NOT NULL,
host TEXT NOT NULL, expires_at INTEGER NOT NULL, activated_at INTEGER, created_at INTEGER NOT NULL
);
CREATE TABLE transcription_devices (
id TEXT PRIMARY KEY, install_id TEXT, name TEXT NOT NULL, lumi_user_id TEXT NOT NULL,
credential_hash TEXT NOT NULL, capabilities_json TEXT NOT NULL, metadata_json TEXT NOT NULL,
first_connected_at INTEGER NOT NULL, last_connected_at INTEGER NOT NULL, revoked_at INTEGER,
pairing_host TEXT
);
`);
const activeId = crypto.randomUUID();
const revokedId = crypto.randomUUID();
const insert = db.prepare("INSERT INTO transcription_devices VALUES (?, NULL, 'PC', 'admin', 'hash', ?, '{}', 1, 1, ?, 'https://lumi.example')");
insert.run(activeId, JSON.stringify(["transcription.capture.v1"]), null);
insert.run(revokedId, JSON.stringify(["transcription.capture.v1"]), 2);
const store = new DeviceStore(db, { now: () => 50 });
assert.equal(store.list().find((device) => device.id === activeId).capabilities.includes("overlay.read.v1"), true,
"the one-time migration must grant active paired devices overlay access");
assert.equal(store.list({ status: "revoked" }).find((device) => device.id === revokedId).capabilities.includes("overlay.read.v1"), false,
"the migration must not modify revoked devices");
store.setCapabilities(activeId, ["transcription.capture.v1"]);
new DeviceStore(db, { now: () => 100 });
assert.equal(store.list().find((device) => device.id === activeId).capabilities.includes("overlay.read.v1"), false,
"rerunning migrations must not restore a capability removed after the one-time grant");
db.close();
}
function verifyLocalhostTransportPolicy() {
assert.equal(normalizeHost("http://localhost:3000/path"), "http://localhost:3000");
assert.equal(normalizeHost("http://127.0.0.1:3000"), "http://127.0.0.1:3000");
@ -310,7 +384,8 @@ async function verifySessionLifecycle() {
}
const provider = new Provider();
const delivered = [];
const deliveryFactory = () => ({ start: async () => {}, stop: async () => {}, pause: async () => {}, resume: async () => {}, deliver: async (event) => { delivered.push(event); return { disposition: "simulated" }; } });
const deliveryModes = [];
const deliveryFactory = () => ({ start: async (options) => { deliveryModes.push(options); }, stop: async () => {}, pause: async () => {}, resume: async () => {}, deliver: async (event) => { delivered.push(event); return { disposition: "simulated" }; } });
const coordinator = new SessionCoordinator({ provider, deliveryFactory, graceMs: 25 });
const { session } = coordinator.create({ id: "device" }, () => {});
const primary = crypto.randomUUID();
@ -328,6 +403,13 @@ async function verifySessionLifecycle() {
await new Promise((resolve) => setTimeout(resolve, 40));
assert.equal(coordinator.status(session.id).state, "idle");
assert.equal(provider.stops, 1);
await coordinator.start(session.id, { mode: "stream_test" });
assert.equal(coordinator.status(session.id).mode, "stream_test");
assert.equal(deliveryModes.at(-1).testMode, true, "private Stream Testing must never use normal live caption delivery");
await coordinator.updateObsState(session.id, { streaming: true, auto_start: true });
assert.equal(coordinator.status(session.id).mode, "stream_test", "OBS startup must not replace the private caption session");
assert.equal((await coordinator.audio(session.id, { ...frame, sequence: 2 })).accepted, true);
await coordinator.stop(session.id, "stream_test_ended");
await coordinator.close();
}
@ -482,7 +564,18 @@ function verifyArtifactsAndLogs(temp) {
manager.installZip(runtimeEntry, runtimeZip);
manager.installZip(runtimeEntry, runtimeZip);
assert.equal(fs.readFileSync(path.join(runtimeRoot, runtimeEntry.id, "bin", "lumi-whisper-worker.exe"), "utf8"), "portable-worker");
assert.equal(sanitize({ pcm: Buffer.alloc(10), device_secret: "secret", stable_text: "hello" }, false).pcm, "[redacted]");
const sanitized = sanitize({
pcm: Buffer.alloc(10),
device_secret: "secret",
webhook_signature: "private-signature",
message: "LumiDevice device-id.private-device-secret Authorization=private-value Cookie: session=private-cookie",
stable_text: "hello"
}, false);
assert.equal(sanitized.pcm, "[redacted]");
assert.equal(sanitized.webhook_signature, "[redacted]");
assert.equal(sanitized.message.includes("private-value"), false);
assert.equal(sanitized.message.includes("private-cookie"), false);
assert.equal(sanitized.message.includes("private-device-secret"), false);
const logsRoot = path.join(temp, "logs");
const logs = new JsonlDiagnosticLog(logsRoot, { retentionDays: 1, maxBytes: 100 });
logs.append({ kind: "caption", stable_text: "hello", pcm: Buffer.alloc(10) });

View File

@ -1,5 +1,11 @@
# Song Overlay changelog
## 0.1.3
- Reuse the Companion device session for Song Overlay traffic instead of asking each installed plugin to authenticate independently.
- Refresh the active overlay when playback changes and keep the OBS pop-out surface transparent and consistent with the editor preview.
- Preserve resilient song-state delivery and the shared announcement template used by the `!music` command.
## 0.1.2
- Added Spotify playback capture through Lumi Companion with shared paired-device authentication.

View File

@ -1,6 +1,7 @@
const crypto = require("crypto");
const fs = require("fs");
const path = require("path");
const { createLogger } = require("../../src/services/logger");
let overlayService = null;
let encryptSecret = null;
let decryptSecret = null;
@ -60,7 +61,7 @@ const DEFAULTS = Object.freeze({
const renderClients = new Set();
const announcementTimers = new Map();
let pluginLogger = console;
let pluginLogger = createLogger("plugin:now_playing", { category: "plugin" });
let activeDb = null;
let activeSettings = null;
let activeClients = null;
@ -80,7 +81,7 @@ module.exports = {
activeClients = { twitchClient, youtubeClient };
placeholderApi = placeholders;
webApi = web;
pluginLogger = logger || console;
pluginLogger = logger || pluginLogger;
fs.mkdirSync(COVER_DIR, { recursive: true });
ensureTables(db);
@ -244,7 +245,9 @@ module.exports = {
if (changedTrack && next.playback_status === "playing") scheduleAnnouncement(db, next.track_key);
return res.json({ accepted: true, server_time: Date.now(), sequence: next.sequence });
} catch (error) {
pluginLogger.warn?.("Rejected Companion song update", { error: error?.message || String(error) });
pluginLogger.warn("Rejected Companion song update", error, {
event: "companion_song_update_rejected"
});
return res.status(error?.status || 400).json({ accepted: false, error: error?.message || "Invalid song update." });
}
});
@ -840,7 +843,9 @@ function scheduleAnnouncement(db, trackKey) {
const current = getStateRow(db);
if (!current || current.track_key !== trackKey || current.playback_status !== "playing") return;
announceTrack({ db, state: current }).catch((error) => {
pluginLogger.warn?.("Song Overlay chat announcement failed", { error: error?.message || String(error) });
pluginLogger.warn("Song Overlay chat announcement failed", error, {
event: "chat_announcement_failed"
});
});
}, 1800);
timer.unref?.();

View File

@ -1,7 +1,7 @@
{
"id": "now_playing",
"name": "Song Overlay",
"version": "0.1.2",
"version": "0.1.3",
"description": "Provider-neutral song state, chat announcements, and a customizable Lumi Overlay source.",
"main": "index.js",
"channel": "stable",

View File

@ -2,11 +2,12 @@ const crypto = require("crypto");
const express = require("express");
const path = require("path");
const discord = require("discord.js");
const { log } = require("../../src/services/logger");
const { createLogger } = require("../../src/services/logger");
const { getPlatformStatus } = require("../../src/services/platforms");
const placeholderService = require("../../src/services/placeholders");
const PLUGIN_ID = "throne_wishlist";
let pluginLogger = createLogger(`plugin:${PLUGIN_ID}`, { category: "plugin" });
const NAMESPACE = "throne";
const REPLAY_WINDOW_SECONDS = 300;
const DEBUG_IDLE_MS = 300000;
@ -74,8 +75,10 @@ module.exports = {
settings,
discordClient,
twitchClient,
youtubeClient
youtubeClient,
logger
}) {
pluginLogger = logger || pluginLogger;
ensureTables(db);
ensureDefaults(db);
registerPlaceholderSupport(placeholders);
@ -446,17 +449,19 @@ function humanizePlaceholderLabel(value) {
function registerStoredEndpoints({ db, webhookApi, clients }) {
if (!webhookApi) {
log("warn", "Throne Wishlist webhook framework unavailable");
pluginLogger.warn("Throne Wishlist webhook framework unavailable", null, {
event: "webhook_framework_unavailable"
});
return;
}
for (const endpoint of listEndpoints(db).filter((entry) => entry.enabled)) {
try {
registerEndpoint({ db, webhookApi, endpoint, clients });
} catch (error) {
log("error", "Throne endpoint registration failed", {
endpointId: endpoint.id,
message: error?.message || String(error)
});
pluginLogger.error("Throne endpoint registration failed", {
endpoint_id: endpoint.id,
error
}, { event: "endpoint_registration_failed" });
}
}
}
@ -555,7 +560,7 @@ async function handleIncoming({
"SELECT event_id FROM throne_event_deliveries WHERE event_id = ?"
).get(eventId);
if (existing) {
log("info", `Webhook - Throne - ${EVENT_TYPES.includes(eventType) ? eventType : "unknown"}`, JSON.stringify({
pluginLogger.info("Duplicate Throne webhook ignored", {
provider: "throne",
authentic: true,
authenticity_status: "duplicate",
@ -564,7 +569,7 @@ async function handleIncoming({
event_id: eventId,
event_type: eventType,
received_at: new Date(context.receivedAt).toISOString()
}));
}, { event: "webhook_duplicate_ignored" });
return { status: 204 };
}
db.prepare(
@ -581,10 +586,10 @@ async function handleIncoming({
receivedAt: context.receivedAt,
clients
}).catch((error) => {
log("error", "Throne platform delivery failed", {
eventId,
message: error?.message || String(error)
});
pluginLogger.error("Throne platform delivery failed", {
event_id: eventId,
error
}, { event: "platform_delivery_failed" });
});
});
return { status: 202, body: { accepted: true } };
@ -669,11 +674,11 @@ async function sendPlatformMessages({ db, payload, endpoint, receivedAt, clients
eventId
);
for (const failure of failed) {
log("warn", "Throne destination send failed", {
eventId,
pluginLogger.warn("Throne destination send failed", {
event_id: eventId,
platform: failure.platform,
message: failure.error
});
}, { event: "destination_send_failed" });
}
}
@ -815,7 +820,7 @@ function writePayloadLog({
: EVENT_TYPES.includes(eventType)
? eventType
: "unknown";
log("info", `Webhook - Throne - ${logEventType}`, JSON.stringify({
pluginLogger.info("Throne webhook received", {
provider: "throne",
authentic: verification.authentic,
authenticity_status: authenticityStatus,
@ -824,10 +829,10 @@ function writePayloadLog({
event_id: eventId,
event_type: eventType,
received_at: new Date(context.receivedAt).toISOString(),
payload: payload || null,
raw_body_preview: context.jsonError ? context.rawBodyText.slice(0, 4000) : null,
signature_prefix: (headerValue(context.headers, "x-signature-ed25519") || "").slice(0, 12)
}));
payload_present: Boolean(payload),
payload_key_count: payload && typeof payload === "object" ? Object.keys(payload).length : 0,
raw_body_bytes: Buffer.byteLength(context.rawBody || "")
}, { event: `webhook_${logEventType}` });
}
function buildPayloadPreview({ receivedAt, eventType, eventId, authentic, status }) {

View File

@ -1,11 +1,12 @@
const crypto = require("crypto");
const path = require("path");
const discord = require("discord.js");
const { Permissions } = discord;
const { ensureUserForIdentity } = require("../../src/services/users");
const { log } = require("../../src/services/logger");
const PLUGIN_ID = "welcome_messages";
const discord = require("discord.js");
const { Permissions } = discord;
const { ensureUserForIdentity } = require("../../src/services/users");
const { createLogger } = require("../../src/services/logger");
const PLUGIN_ID = "welcome_messages";
let pluginLogger = createLogger(`plugin:${PLUGIN_ID}`, { category: "plugin" });
const CONFIG_KEY = "config";
const ALLOWED_PLACEHOLDERS = new Set([
"username",
@ -44,10 +45,13 @@ const PRONOUN_SETS = [
module.exports = {
id: PLUGIN_ID,
init({ web, db, discordClient, settings }) {
ensureTables(db);
ensureConfig(db);
log("info", "Welcome Messages plugin initialized");
init({ web, db, discordClient, settings, logger }) {
pluginLogger = logger || pluginLogger;
ensureTables(db);
ensureConfig(db);
pluginLogger.info("Welcome Messages plugin initialized", null, {
event: "plugin_initialized"
});
const router = web.createRouter();
@ -382,40 +386,42 @@ function ensureSidebarNavItem(settings) {
function attachDiscordListener(discordClient, db) {
if (!discordClient || discordClient[LISTENER_KEY]) {
if (!discordClient) {
log("warn", "Welcome Messages listener not attached", {
reason: "Discord client is unavailable"
});
pluginLogger.warn("Welcome Messages listener not attached", {
reason: "Discord client is unavailable"
}, { event: "listener_not_attached" });
}
return;
}
discordClient[LISTENER_KEY] = true;
discordClient.on("guildMemberAdd", (member) => {
handleMemberJoin(member, db).catch((error) => {
log("error", "Welcome Messages join handler failed", error);
pluginLogger.error("Welcome Messages join handler failed", error, {
event: "join_handler_failed"
});
});
});
log("info", "Welcome Messages listener attached", {
event: "guildMemberAdd",
memberIntent: detectGuildMembersIntent(discordClient)
});
pluginLogger.info("Welcome Messages listener attached", {
discord_event: "guildMemberAdd",
member_intent: detectGuildMembersIntent(discordClient)
}, { event: "listener_attached" });
}
async function handleMemberJoin(member, db) {
const config = getConfig(db);
if (!config.enabled || !config.channelId) {
log("debug", "Welcome Messages skipped join", {
reason: !config.enabled ? "Plugin posting disabled" : "No welcome channel configured",
guildId: member?.guild?.id || null,
userId: member?.user?.id || member?.id || null
});
pluginLogger.debug("Welcome Messages skipped join", {
reason: !config.enabled ? "Plugin posting disabled" : "No welcome channel configured",
guild_id: member?.guild?.id || null,
user_id: member?.user?.id || member?.id || null
}, { event: "join_skipped" });
return;
}
const discordId = member?.user?.id || member?.id;
if (!discordId) {
log("warn", "Welcome Messages skipped join", {
reason: "Missing Discord user id",
guildId: member?.guild?.id || null
});
pluginLogger.warn("Welcome Messages skipped join", {
reason: "Missing Discord user id",
guild_id: member?.guild?.id || null
}, { event: "join_skipped" });
return;
}
const existingIdentity = db
@ -427,12 +433,12 @@ async function handleMemberJoin(member, db) {
const profile = ensureDiscordIdentity(member);
const channelResult = await validateTextChannel(member.client, config.channelId);
if (!channelResult.valid || !channelResult.channel) {
log("warn", "Welcome Messages skipped join", {
reason: channelResult.message,
guildId: member?.guild?.id || null,
userId: discordId,
channelId: config.channelId
});
pluginLogger.warn("Welcome Messages skipped join", {
reason: channelResult.message,
guild_id: member?.guild?.id || null,
user_id: discordId,
channel_id: config.channelId
}, { event: "join_skipped" });
return;
}
const pool =
@ -441,11 +447,11 @@ async function handleMemberJoin(member, db) {
: config.welcomeMessages;
const message = chooseMessage(pool) || chooseMessage(config.welcomeMessages);
if (!message) {
log("warn", "Welcome Messages skipped join", {
reason: "No enabled message templates",
guildId: member?.guild?.id || null,
userId: discordId
});
pluginLogger.warn("Welcome Messages skipped join", {
reason: "No enabled message templates",
guild_id: member?.guild?.id || null,
user_id: discordId
}, { event: "join_skipped" });
return;
}
const pronoun = getSubjectPronoun(db, profile?.id);
@ -455,21 +461,21 @@ async function handleMemberJoin(member, db) {
content: rendered,
allowedMentions: { parse: [] }
});
log("info", "Welcome Messages sent welcome message", {
guildId: member?.guild?.id || null,
userId: discordId,
channelId: config.channelId,
returning,
pool: returning && config.welcomeBackEnabled ? "welcomeBackMessages" : "welcomeMessages",
messageId: message.id
});
} catch (error) {
log("error", "Welcome Messages failed to send message", {
guildId: member?.guild?.id || null,
userId: discordId,
channelId: config.channelId,
error: error?.stack || error?.message || String(error)
});
pluginLogger.info("Welcome Messages sent welcome message", {
guild_id: member?.guild?.id || null,
user_id: discordId,
channel_id: config.channelId,
returning,
pool: returning && config.welcomeBackEnabled ? "welcomeBackMessages" : "welcomeMessages",
message_id: message.id
}, { event: "welcome_message_sent" });
} catch (error) {
pluginLogger.error("Welcome Messages failed to send message", {
guild_id: member?.guild?.id || null,
user_id: discordId,
channel_id: config.channelId,
error
}, { event: "welcome_message_send_failed" });
}
}
@ -638,21 +644,23 @@ async function buildDiagnostics(discordClient, channelId) {
function logStartupDiagnostics(discordClient) {
const memberIntent = detectGuildMembersIntent(discordClient);
const details = {
discordClientAvailable: Boolean(discordClient),
discordClientReady: Boolean(discordClient?.readyAt),
memberIntent
};
if (memberIntent === "configured") {
log("info", "Welcome Messages diagnostics passed", details);
return;
}
log("warn", "Welcome Messages diagnostics warning", {
...details,
message:
"Guild member join events require the runtime Discord client to start with GuildMembers/GUILD_MEMBERS and the Discord Developer Portal Server Members Intent enabled."
});
}
const details = {
discord_client_available: Boolean(discordClient),
discord_client_ready: Boolean(discordClient?.readyAt),
member_intent: memberIntent
};
if (memberIntent === "configured") {
pluginLogger.info("Welcome Messages diagnostics passed", details, {
event: "startup_diagnostics_passed"
});
return;
}
pluginLogger.warn("Welcome Messages diagnostics warning", {
...details,
message:
"Guild member join events require the runtime Discord client to start with GuildMembers/GUILD_MEMBERS and the Discord Developer Portal Server Members Intent enabled."
}, { event: "startup_diagnostics_warning" });
}
function detectGuildMembersIntent(discordClient) {
if (!discordClient) {

View File

@ -2,6 +2,394 @@
"schema_version": 1,
"channel": "stable",
"releases": [
{
"version": "0.3.11",
"ref": "refs/tags/v0.3.11",
"released_at": "2026-07-29",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Releases Companion 0.2.8 with captured hotkeys, multi-source Windows media discovery, and an interactive native overlay editor; improves tenure presentation and adds per-user command and expression favorites. The additive command-usage table preserves all existing settings, identities, statistics, intervals, plugins, pairing records, databases, uploads, models, and secrets.",
"plugins": {
"auto-vc": "0.1.7",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_overlay": "0.1.1",
"lumi_transcription": "0.2.8",
"moderation": "0.1.5",
"now_playing": "0.1.3",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.10",
"ref": "refs/tags/v0.3.10",
"released_at": "2026-07-27",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Adds reusable platform-tenure interval history for Twitch, YouTube, and Discord; contextual current and total duration placeholders with deterministic formatting; safe platform reconciliation; admin diagnostics; and Platform tenure sections on Stats and Leaderboards. Existing settings, identities, statistics, intervals, plugins, pairing records, databases, uploads, models, and secrets remain preserved.",
"plugins": {
"auto-vc": "0.1.7",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_overlay": "0.1.1",
"lumi_transcription": "0.2.7",
"moderation": "0.1.5",
"now_playing": "0.1.3",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.9",
"ref": "refs/tags/v0.3.9",
"released_at": "2026-07-26",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Restores animated emotes, Twitch badge artwork, and Discord GIF media through the existing OBS and native Companion overlay paths; adds persistent Auto VC welcome controls and timed ownership claiming; and releases Companion 0.2.7. Existing settings, lobbies, rooms, owners, permissions, pairing identities, plugin data, OBS configuration, databases, uploads, models, and secrets remain preserved.",
"plugins": {
"auto-vc": "0.1.7",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_overlay": "0.1.1",
"lumi_transcription": "0.2.7",
"moderation": "0.1.5",
"now_playing": "0.1.3",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.8",
"ref": "refs/tags/v0.3.8",
"released_at": "2026-07-26",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Adds the native Windows Lumi Overlay through the existing paired-device and provider pipelines, preserves active and revoked device semantics, and releases Companion 0.2.6 with draft-safe settings, direct single-page plugin navigation, and clearer overlay controls. Existing settings, pairing identities, plugin data, OBS configuration, databases, uploads, models, and secrets remain preserved.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_overlay": "0.1.0",
"lumi_transcription": "0.2.6",
"moderation": "0.1.5",
"now_playing": "0.1.3",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.7",
"ref": "refs/tags/v0.3.7",
"released_at": "2026-07-26",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Moves production Stream Testing TLS termination to Nginx Proxy Manager while preserving RTMPS Companion destinations, plain local MediaMTX ingest, localhost development, session credentials, and all existing operator data. Adds DB-backed public and local ports plus external TLS and RTMP route validation, and removes unused Lumi ACME and DNS automation without deleting stored data.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_transcription": "0.2.5",
"moderation": "0.1.5",
"now_playing": "0.1.3",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.6",
"ref": "refs/tags/v0.3.6",
"released_at": "2026-07-26",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Adds encrypted Domeneshop DNS-01 automation for RTMPS certificate issuance and renewal when OpenResty or another HTTPS reverse proxy owns the HTTP challenge path. Lumi creates, verifies, and removes short-lived DNS records without external packages or certificate paths. Existing local and plugin data remains preserved.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_transcription": "0.2.5",
"moderation": "0.1.5",
"now_playing": "0.1.3",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.5",
"ref": "refs/tags/v0.3.5",
"released_at": "2026-07-26",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Adds resilient Companion reconnect and caption controls, progressive private-test captions, improved Companion navigation and stream-viewer access, automatic Lumi-managed RTMPS certificates, refreshed Song Overlay delivery, and repository-wide durable redacted logging. Existing local and plugin data remains preserved.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_transcription": "0.2.4",
"moderation": "0.1.5",
"now_playing": "0.1.3",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.4",
"ref": "refs/tags/v0.3.4",
"released_at": "2026-07-26",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Fixes Companion package generation behind Lumi's private HTTPS reverse proxy, applies one guarded transport policy to HTTP and WebSocket device traffic, and standardizes durable redacted operational logging. Existing local and plugin data remains preserved.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_transcription": "0.2.3",
"moderation": "0.1.5",
"now_playing": "0.1.2",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.3",
"ref": "refs/tags/v0.3.3",
"released_at": "2026-07-25",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Replaces the FFmpeg Stream Testing receiver with Lumi-managed checksum-pinned MediaMTX source remuxing, authenticated same-origin playback, and factual receiver diagnostics. Existing local and plugin data remains preserved.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_transcription": "0.2.3",
"moderation": "0.1.5",
"now_playing": "0.1.2",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.2",
"ref": "refs/tags/v0.3.2",
"released_at": "2026-07-24",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Fixes the managed OBS Bridge streaming-service ownership crash and requires the corrected bridge before another private Stream Test. Existing local and plugin data remains preserved.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_transcription": "0.2.2",
"moderation": "0.1.5",
"now_playing": "0.1.2",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.1",
"ref": "refs/tags/v0.3.1",
"released_at": "2026-07-24",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Fixes Stream Testing startup feedback and Windows FFmpeg discovery, and adds explicit loaded OBS Bridge version checks with update actions. Existing local and plugin data remains preserved.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_transcription": "0.2.1",
"moderation": "0.1.5",
"now_playing": "0.1.2",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.3.0",
"ref": "refs/tags/v0.3.0",
"released_at": "2026-07-24",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Adds authenticated private OBS stream testing with crash-safe service restoration, adaptive temporary playback, corrected overlay media behavior, and repeatable Companion updates. Existing local and plugin data remains preserved.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.5",
"lumi_transcription": "0.2.0",
"moderation": "0.1.5",
"now_playing": "0.1.2",
"okf": "0.1.2",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{
"version": "0.2.27",
"ref": "refs/tags/v0.2.27",

Some files were not shown because too many files have changed in this diff Show More