Enhance structured logging across Lumi

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Lumi changelog
 
+## 0.2.11
+
+- Added structured component, activity, event, and request metadata across Core, plugins, platforms, commands, webhooks, diagnostics, and WebUI administrator actions.
+- Added recursive credential redaction, bounded log payloads, automatic age/count retention, and a scoped logger supplied to plugins.
+- Upgraded Admin Logs with full server-side search and filters, filtered totals, live entries, clearer metadata, configurable retention, and text or JSON Lines downloads.
+
 ## 0.2.10
 
 - Fixed local Windows startup under Node.js 24 by invoking npm through its JavaScript CLI instead of directly spawning `npm.cmd`.

README.md

@@ -25,6 +25,14 @@ Before starting Lumi, you can check the runtime and native dependency with
 `npm run verify:preflight`. Run the complete syntax and focused test suite with
 `npm run verify:all`; it stops at the first failure and prints the failing check.
 
+## Logs
+
+Administrators can review structured Core, plugin, platform, command, WebUI,
+security, and integration activity under **Admin → Logs**. Logs support live
+updates, component/activity filters, server-side search, redacted details,
+request IDs, retention limits, and text or JSON Lines downloads. Plugin and
+core logging conventions are documented in [`docs/logging.md`](docs/logging.md).
+
 You can also seed local configuration with a `.env` file. Use `.env.example`
 as the template; `.env` is ignored by git.
 

TODO.md

@@ -677,6 +677,7 @@ This section is for Lumi chat answer feedback and AI Improvement Center work, no
 
 ## Done
 
+- 2026-07-18: Reworked shared logging repo-wide with structured component/category/event/request metadata, recursive secret redaction, bounded details, automatic retention, scoped plugin loggers, platform/command/webhook/lifecycle/admin HTTP auditing, live `/admin/logs` updates, complete server-side filters and summaries, and text/JSONL exports.
 - 2026-07-18: Fixed core 0.2.10 local Windows startup under Node.js 24 by resolving npm's JavaScript CLI through the active Node installation, retaining a command-shell fallback, and verifying automatic dependency repair through the real launcher on port 3000.
 - 2026-07-18: Fixed core 0.2.9 and Lumi AI 0.8.5 forms whose `action` or `method` fields could shadow the form endpoint, added a shared clobber-safe resolver, and migrated the affected feedback/settings requests to it.
 - 2026-07-18: Fixed Lumi AI 0.8.4 Improvement Center state changes for no-change, dismiss, edit, and restore actions with an explicit authenticated JSON contract, actionable errors, safe diagnostics, and focused regression coverage.

docs/logging.md

@@ -0,0 +1,46 @@
+# Logging
+
+Lumi stores core, plugin, integration, command, and administrator activity in the shared SQLite `logs` table. Administrators review it at **Admin → Logs**. The page supports severity, component, activity type, range, full server-side search, and entry-count filters. New matching entries arrive through Lumi's authenticated event stream without a page refresh.
+
+Each entry may include:
+
+- `level`: `debug`, `info`, `warn`, or `error`.
+- `source`: the component that produced the entry, such as `core:web`, `core:commands`, `platform:twitch`, or `plugin:lumi_ai`.
+- `category`: a broad activity type such as `lifecycle`, `audit`, `http`, `command`, `integration`, `security`, or `plugin`.
+- `event`: a stable machine-readable event name.
+- `request_id`: the matching HTTP or diagnostic request identifier when available.
+- A short message and optional structured details.
+
+## Writing logs
+
+Existing `log(level, message, details)` calls remain supported. New core services should use a scoped logger:
+
+```js
+const { createLogger } = require("./logger");
+const logger = createLogger("core:example", { category: "integration" });
+
+logger.info("Example connected", { endpoint: "primary" }, { event: "example_ready" });
+logger.error("Example request failed", error, { event: "example_failed" });
+```
+
+Every loaded plugin receives a logger scoped to `plugin:<plugin-id>` in its `init` context:
+
+```js
+init({ logger }) {
+  logger.info("Plugin feature ready", { mode: "automatic" }, { event: "feature_ready" });
+}
+```
+
+Use `debug` for high-volume successful activity, `info` for meaningful state changes, `warn` for degraded or rejected work, and `error` for failed work that needs attention. Do not log full chat messages, request bodies, credentials, session data, third-party response bodies, or user access tokens.
+
+The shared logger recursively redacts credential-shaped object keys, authorization headers, and common secret query parameters before storage. Redaction is a final safety boundary, not permission to pass known secrets to the logger. Messages and details are length-bounded so a malformed response cannot grow the database without limit.
+
+## Administrator actions and requests
+
+Mutating WebUI requests produce an `audit` entry with method, matched route, result status, duration, role, user ID, and a request ID. Failed and unusually slow requests are recorded as `http` warnings or errors. Query strings and submitted bodies are not included. Webhook, platform, diagnostics, plugin lifecycle, command failure, startup, and shutdown paths use dedicated sources and events.
+
+## Retention and downloads
+
+By default Lumi retains logs for 30 days and keeps at most 100,000 entries. Administrators can change both limits under **Log storage** on the logs page. Cleanup runs at startup and immediately after retention settings are saved. It only removes rows from the shared logs table.
+
+Filtered downloads are available as readable text or JSON Lines. JSON Lines preserves all structured fields for external diagnostics without changing Lumi's database.

docs/lumi-ui.md

@@ -182,10 +182,13 @@ uptime, memory, plugin counts, content counts, and recent log severity totals.
 The dashboard renders lightweight SVG graphs using Lumi tokens and does not add
 a frontend framework dependency.
 
-The logs page keeps server-side range/severity/limit filters and adds a labeled
+The logs page provides server-side range, severity, component, activity type,
-responsive filter bar with search, reset, refresh, and download actions. Search
+search, and limit filters in a responsive control bar. Summary totals reflect
-filters the loaded entries client-side; changing range, severity, or limit
+the complete filtered result rather than only the visible page. Entries expose
-reloads the same `/admin/logs` route with query parameters.
+their source, category, stable event name, and request ID, and matching events
+arrive live through the existing authenticated event stream. Downloads retain
+the active component/activity/search filters and support readable text or JSON
+Lines. Retention controls remain collapsed until needed.
 
 ## Updates And Local-Only Files
 

knowledge/core/lumi-core.md

@@ -14,7 +14,7 @@ editable: false
 Lumi is the core web UI and bot runtime.
 ## Runtime
 Package: lumi-bot
-Version: 0.2.10
+Version: 0.2.11
 ## Routes
 - GET /api/events
 - POST /api/destructive-confirmations
@@ -91,6 +91,7 @@ Version: 0.2.10
 - POST /admin/theming/custom/:id/delete
 - POST /admin/theming
 - GET /admin/logs
+- POST /admin/logs/retention
 - GET /admin/logs/download
 - GET /admin/feedback
 - POST /admin/feedback/export
@@ -815,16 +816,25 @@ Version: 0.2.10
 ### GET /admin/logs
 
 - Purpose: Displays, downloads, or manages application logs.
-- Inputs: query: `level`, `limit`, `range`
+- Inputs: query: `category`, `level`, `limit`, `q`, `range`, `source`
 - Response format: HTML page rendered from an EJS view
 - Access: admin access expected
 - Side effects: Usually read-only.
 - Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations.
 
+### POST /admin/logs/retention
+
+- Purpose: Displays, downloads, or manages application logs.
+- Inputs: body: `max_age_days`, `max_entries`
+- Response format: HTTP redirect after handling the request
+- Access: admin access expected; logged-in session required or used
+- Side effects: writes or mutates server-side state
+- Limits/notes: Generated from static route source analysis; confirm exact behavior in the handler before changing integrations. Most non-API POST routes are browser form submissions and usually redirect after completion.
+
 ### GET /admin/logs/download
 
 - Purpose: Displays, downloads, or manages application logs.
-- Inputs: query: `level`, `limit`, `range`
+- Inputs: query: `category`, `format`, `level`, `limit`, `q`, `range`, `source`
 - Response format: plain or HTML response
 - Access: admin access expected
 - Side effects: writes or mutates server-side state

package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "lumi-bot",
-  "version": "0.2.10",
+  "version": "0.2.11",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "lumi-bot",
-      "version": "0.2.10",
+      "version": "0.2.11",
       "dependencies": {
         "adm-zip": "^0.5.12",
         "better-sqlite3": "^11.5.0",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "lumi-bot",
-  "version": "0.2.10",
+  "version": "0.2.11",
   "private": true,
   "type": "commonjs",
   "scripts": {

release-index.json

@@ -2,6 +2,36 @@
   "schema_version": 1,
   "channel": "stable",
   "releases": [
+    {
+      "version": "0.2.11",
+      "ref": "refs/tags/v0.2.11",
+      "released_at": "2026-07-18",
+      "installable": true,
+      "rollback_safe": true,
+      "replaces_versions": [
+        "1.2.0"
+      ],
+      "data_policy": "preserve",
+      "dependency_policy": "sync_on_restart",
+      "migration_notes": "Adds structured, redacted, bounded logging and new optional log metadata columns. Existing logs and all settings, databases, plugin data, models, uploads, feedback, and secrets are preserved.",
+      "plugins": {
+        "auto-vc": "0.1.6",
+        "birthday": "0.1.3",
+        "economy-framework": "0.2.10",
+        "economy-games": "0.1.7",
+        "expression-interaction": "0.2.1",
+        "lumi_ai": "0.8.5",
+        "moderation": "0.1.5",
+        "okf": "0.1.1",
+        "quotes": "0.1.2",
+        "sample-plugin": "0.1.0",
+        "throne_wishlist": "0.1.2",
+        "welcome_messages": "0.1.1"
+      },
+      "tools": {
+        "lumi_ai_web_search": "0.1.1"
+      }
+    },
     {
       "version": "0.2.10",
       "ref": "refs/tags/v0.2.10",

scripts/verify-all.js

@@ -8,6 +8,7 @@ const checks = [
   "scripts/verify-webui.js",
   "scripts/verify-web-auth.js",
   "scripts/verify-feedback-system.js",
+  "scripts/verify-logging.js",
   "scripts/verify-placeholders.js",
   "scripts/verify-release-metadata.js",
   "scripts/verify-update-system.js",

scripts/verify-logging.js

@@ -0,0 +1,85 @@
+const assert = require("assert");
+const fs = require("fs");
+const path = require("path");
+
+const root = path.join(__dirname, "..");
+const sandbox = fs.mkdtempSync(path.join(root, ".tmp-lumi-logging-"));
+const serviceDir = path.join(sandbox, "src", "services");
+fs.mkdirSync(serviceDir, { recursive: true });
+for (const file of ["db.js", "logger.js", "web-events.js"]) {
+  fs.copyFileSync(path.join(root, "src", "services", file), path.join(serviceDir, file));
+}
+
+let database;
+try {
+  database = require(path.join(serviceDir, "db.js"));
+  database.db.exec(`CREATE TABLE logs (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    level TEXT NOT NULL,
+    message TEXT NOT NULL,
+    details TEXT,
+    created_at INTEGER NOT NULL
+  )`);
+  database.migrate();
+  const logger = require(path.join(serviceDir, "logger.js"));
+  const { db } = database;
+  const columns = db.prepare("PRAGMA table_info(logs)").all().map((column) => column.name);
+  for (const column of ["source", "category", "event", "request_id"]) {
+    assert(columns.includes(column), `logs.${column} migration`);
+  }
+
+  logger.log("info", "Legacy-compatible entry", { status: "ready" });
+  const pluginLog = logger.createLogger("plugin:example", { category: "plugin" });
+  pluginLog.warn("Example warning", {
+    access_token: "must-not-be-stored",
+    nested: { password: "also-secret", safe: "visible" },
+    url: "https://example.com/run?token=hidden&mode=safe"
+  }, { event: "example_warning", requestId: "request-123" });
+  logger.withLogContext({ source: "core:test", category: "verification", event: "context_entry" }, () => {
+    logger.log("debug", "Context-aware entry", "authorization=private-value");
+  });
+
+  const all = logger.listLogs({ limit: 20 });
+  assert.equal(all.length, 3);
+  const warning = all.find((entry) => entry.event === "example_warning");
+  assert.equal(warning.source, "plugin:example");
+  assert.equal(warning.category, "plugin");
+  assert.equal(warning.request_id, "request-123");
+  assert(warning.details.includes("[REDACTED]"));
+  assert.equal(warning.details.includes("must-not-be-stored"), false);
+  assert.equal(warning.details.includes("also-secret"), false);
+  assert.equal(warning.details.includes("token=hidden"), false);
+  assert(warning.details.includes("visible"));
+
+  assert.equal(logger.listLogs({ sources: ["plugin:example"] }).length, 1);
+  assert.equal(logger.listLogs({ categories: ["verification"] }).length, 1);
+  assert.equal(logger.listLogs({ search: "request-123" }).length, 1);
+  assert.equal(logger.listLogs({ levels: ["error"] }).length, 0);
+  const summary = logger.summarizeLogs({});
+  assert.equal(summary.total, 3);
+  assert.equal(summary.levels.warn, 1);
+  assert(logger.listLogFacets().sources.some((item) => item.value === "plugin:example" && item.count === 1));
+
+  db.prepare("UPDATE logs SET created_at = ? WHERE message = ?").run(
+    Date.now() - 45 * 24 * 60 * 60 * 1000,
+    "Legacy-compatible entry"
+  );
+  const cleanup = logger.cleanupLogs({ maxAgeDays: 30, maxEntries: 1000 });
+  assert.equal(cleanup.expired, 1);
+  assert.equal(logger.listLogs({ search: "Legacy-compatible" }).length, 0);
+
+  const logView = fs.readFileSync(path.join(root, "src", "web", "views", "admin-logs.ejs"), "utf8");
+  assert.match(logView, /data-log-source/);
+  assert.match(logView, /data-log-category/);
+  assert.match(logView, /data-log-live-status/);
+  assert.match(logView, /name="format"/);
+  const serverSource = fs.readFileSync(path.join(root, "src", "web", "server.js"), "utf8");
+  assert.match(serverSource, /app\.post\("\/admin\/logs\/retention"/);
+  assert.match(serverSource, /summarizeLogs\(query\)/);
+  assert.match(serverSource, /"admin_action"/);
+
+  console.log("Logging verification passed: structured scope, redaction, search, summaries, retention, admin audit, and live UI wiring.");
+} finally {
+  try { database?.db?.close(); } catch {}
+  fs.rmSync(sandbox, { recursive: true, force: true });
+}

scripts/verify-release-metadata.js

@@ -4,8 +4,8 @@ const path = require("path");
 const { findSafeTarget } = require("../src/services/versioning");
 
 const root = path.join(__dirname, "..");
-const releaseVersion = "0.2.10";
+const releaseVersion = "0.2.11";
-const previousCoreVersion = "0.2.9";
+const previousCoreVersion = "0.2.10";
 const earliestCompatibleCoreVersion = "0.1.9";
 const changedPlugins = {
   "auto-vc": { from: "0.1.5", to: "0.1.6", knowledge: "auto-vc" },
@@ -87,4 +87,4 @@ assert.equal(webSearch.minimum_lumi_version, "0.2.0");
 assert.equal(webSearch.minimum_lumi_ai_version, "0.8.2");
 assert.equal(hasVersionHeading(readText("plugins/lumi_ai_web_search/CHANGELOG.md"), webSearch.version), true);
 
-console.log("Release metadata verification passed: core 0.2.10, Lumi AI 0.8.5, and synchronized package metadata.");
+console.log("Release metadata verification passed: core 0.2.11, Lumi AI 0.8.5, and synchronized package metadata.");

scripts/verify-update-system.js

@@ -16,7 +16,7 @@ function readJson(relativePath) {
 
 const releaseIndex = readJson("release-index.json");
 const releaseVersions = releaseIndex.releases.map((release) => release.version);
-assert.deepEqual(releaseVersions, ["0.2.10", "0.2.9", "0.2.8", "0.2.7", "0.2.6", "0.2.5", "0.2.4", "0.2.3", "0.2.2", "0.2.1", "0.2.0", "0.1.9"]);
+assert.deepEqual(releaseVersions, ["0.2.11", "0.2.10", "0.2.9", "0.2.8", "0.2.7", "0.2.6", "0.2.5", "0.2.4", "0.2.3", "0.2.2", "0.2.1", "0.2.0", "0.1.9"]);
 assert.equal(new Set(releaseVersions).size, releaseVersions.length, "release versions must be unique");
 for (const release of releaseIndex.releases) {
   assert.equal(normalizeRepositoryRef(release.ref), release.ref);
@@ -37,6 +37,7 @@ for (const [toolId, version] of Object.entries(currentRelease.tools)) {
 const baseTarget = {
   current_version: "0.2.4",
   available_versions: [
+    { version: "0.2.11", ref: "refs/tags/v0.2.11", rollback_safe: true },
     { version: "0.2.10", ref: "refs/tags/v0.2.10", rollback_safe: true },
     { version: "0.2.9", ref: "refs/tags/v0.2.9", rollback_safe: true },
     { version: "0.2.8", ref: "refs/tags/v0.2.8", rollback_safe: true },
@@ -70,7 +71,7 @@ const corrected = buildStatus({
   channel: "stable"
 });
 assert.equal(corrected.version_correction, true);
-assert.equal(corrected.safe_target_version, "0.2.10");
+assert.equal(corrected.safe_target_version, "0.2.11");
 assert.equal(corrected.update_available, true);
 assert.equal(corrected.blocked, false);
 

src/main.js

@@ -28,6 +28,17 @@ async function main() {
   ensureDefaults();
   registerCorePlaceholders();
   logger.hookConsole();
+  const runtimeLog = logger.createLogger("core:runtime", { category: "lifecycle" });
+  const logCleanup = logger.cleanupLogs({
+    maxAgeDays: getSetting("log_retention_days", logger.DEFAULT_MAX_AGE_DAYS),
+    maxEntries: getSetting("log_retention_max_entries", logger.DEFAULT_MAX_ENTRIES)
+  });
+  runtimeLog.info("Lumi startup initiated", {
+    version: require("../package.json").version,
+    node: process.version,
+    platform: process.platform,
+    log_entries_removed: logCleanup.removed
+  }, { event: "startup" });
   try {
     cleanupSnapshots();
   } catch (error) {
@@ -95,7 +106,9 @@ async function main() {
 
   const port = Number(process.env.PORT || 3000);
   app.listen(port, () => {
-    console.log(`WebUI listening on http://localhost:${port}`);
+    runtimeLog.run({ event: "web_ready" }, () => {
+      console.log(`WebUI listening on http://localhost:${port}`, { port });
+    });
   });
 
   const autoUpdateEnabled = getSetting("auto_update_enabled", false);
@@ -127,6 +140,7 @@ async function main() {
       return;
     }
     shuttingDown = true;
+    runtimeLog.info("Lumi shutdown started", null, { event: "shutdown" });
     await overlayConnectorManager.stop();
     await stopPlugins();
     await stopBot();

src/services/command-router.js

@@ -8,6 +8,9 @@ const {
 const { normalizeRandomReplies, selectRandomReply } = require("./command-random");
 const { getEnabledPlatformIds, normalizePlatformSelection } = require("./platforms");
 const placeholders = require("./placeholders");
+const { createLogger } = require("./logger");
+
+const commandLog = createLogger("core:commands", { category: "command" });
 
 function createCommandRouter({ settings }) {
   const commandMap = new Map();
@@ -101,6 +104,11 @@ function createCommandRouter({ settings }) {
     });
     if (customHandled) {
       incrementCommands(user.id);
+      commandLog.debug("Custom command completed", {
+        trigger,
+        platform,
+        user_id: user.id
+      }, { event: "custom_command_completed" });
       return true;
     }
 
@@ -112,15 +120,33 @@ function createCommandRouter({ settings }) {
           await safeReply(reply, result);
           recordCommandUsage(handler.commandId);
           incrementCommands(user.id);
+          commandLog.debug("Command completed", {
+            command_id: handler.commandId,
+            trigger,
+            platform,
+            user_id: user.id
+          }, { event: "command_completed" });
           return true;
         }
         if (result === true) {
           recordCommandUsage(handler.commandId);
           incrementCommands(user.id);
+          commandLog.debug("Command completed", {
+            command_id: handler.commandId,
+            trigger,
+            platform,
+            user_id: user.id
+          }, { event: "command_completed" });
           return true;
         }
       } catch (error) {
-        console.error("Command handler failed", error);
+        commandLog.error("Command handler failed", {
+          command_id: handler.commandId,
+          trigger,
+          platform,
+          user_id: user.id,
+          error
+        }, { event: "command_failed" });
         await safeReply(reply, "Command failed to execute.");
         return true;
       }
@@ -214,7 +240,12 @@ async function handleCustomCommand({ trigger, platform, ctx, raw, reply }) {
     recordCommandUsage(`custom:${trigger}`);
     return true;
   } catch (error) {
-    console.error("Failed to reply to command", error);
+    commandLog.error("Custom command failed", {
+      trigger,
+      platform,
+      user_id: ctx.user.id,
+      error
+    }, { event: "custom_command_failed" });
     await safeReply(reply, "Command failed to execute.");
     return true;
   }
@@ -251,7 +282,7 @@ async function safeReply(reply, content) {
   try {
     await reply(content);
   } catch (error) {
-    console.error("Command reply failed", error);
+    commandLog.error("Command reply failed", error, { event: "command_reply_failed" });
   }
 }
 

src/services/db.js

@@ -150,6 +150,10 @@ function migrate() {
       level TEXT NOT NULL,
       message TEXT NOT NULL,
       details TEXT,
+      source TEXT NOT NULL DEFAULT 'core',
+      category TEXT NOT NULL DEFAULT 'general',
+      event TEXT,
+      request_id TEXT,
       created_at INTEGER NOT NULL
     );
 
@@ -412,6 +416,25 @@ function migrate() {
     db.exec("ALTER TABLE overlays ADD COLUMN canvas_height INTEGER NOT NULL DEFAULT 1080");
   }
 
+  const logColumns = db
+    .prepare("PRAGMA table_info(logs)")
+    .all()
+    .map((column) => column.name);
+  if (!logColumns.includes("source")) {
+    db.exec("ALTER TABLE logs ADD COLUMN source TEXT NOT NULL DEFAULT 'core'");
+  }
+  if (!logColumns.includes("category")) {
+    db.exec("ALTER TABLE logs ADD COLUMN category TEXT NOT NULL DEFAULT 'general'");
+  }
+  if (!logColumns.includes("event")) {
+    db.exec("ALTER TABLE logs ADD COLUMN event TEXT");
+  }
+  if (!logColumns.includes("request_id")) {
+    db.exec("ALTER TABLE logs ADD COLUMN request_id TEXT");
+  }
+  db.exec("CREATE INDEX IF NOT EXISTS logs_source_created_at_idx ON logs (source, created_at)");
+  db.exec("CREATE INDEX IF NOT EXISTS logs_category_created_at_idx ON logs (category, created_at)");
+
   migrateLegacyUsers();
 }
 

src/services/discord.js

@@ -7,6 +7,9 @@ const Partials = discord.Partials;
 const { getSetting, setSetting } = require("./settings");
 const { incrementMessages } = require("./stats");
 const { ensureUserForIdentity } = require("./users");
+const { createLogger } = require("./logger");
+
+const discordLog = createLogger("platform:discord", { category: "integration" });
 
 let client = null;
 
@@ -29,10 +32,10 @@ async function startBot({ commandRouter } = {}) {
   if (intents.length) {
     options.intents = intents;
   }
-  console.log("Discord bot starting with intents", {
+  discordLog.info("Discord bot starting", {
     intents,
     guildMembers: Boolean(resolveIntent("GuildMembers", "GUILD_MEMBERS"))
-  });
+  }, { event: "platform_starting" });
   if (Partials?.Channel) {
     options.partials = [Partials.Channel];
   }
@@ -40,7 +43,7 @@ async function startBot({ commandRouter } = {}) {
   client = new Client(options);
 
   client.on("ready", () => {
-    console.log(`Discord bot ready: ${client.user?.tag}`);
+    discordLog.info("Discord bot ready", { account: client.user?.tag || null }, { event: "platform_ready" });
     const avatarUrl = getBotAvatarUrl(client.user);
     if (avatarUrl) {
       setSetting("bot_avatar_url", avatarUrl);
@@ -86,7 +89,7 @@ async function startBot({ commandRouter } = {}) {
           try {
             await message.reply(content);
           } catch (error) {
-            console.error("Discord command reply failed", error);
+            discordLog.error("Discord command reply failed", error, { event: "reply_failed" });
           }
         }
       });

src/services/logger.js

@@ -1,143 +1,321 @@
+const { AsyncLocalStorage } = require("async_hooks");
 const util = require("util");
 const { db } = require("./db");
 
 const LEVELS = new Set(["debug", "info", "warn", "error"]);
+const DEFAULT_MAX_AGE_DAYS = 30;
+const DEFAULT_MAX_ENTRIES = 100000;
+const MAX_MESSAGE_LENGTH = 1000;
+const MAX_DETAILS_LENGTH = 64 * 1024;
+const contextStorage = new AsyncLocalStorage();
 let consoleHooked = false;
 
 function log(level, ...args) {
+  return writeLog(level, args, contextStorage.getStore() || {});
+}
+
+function createLogger(source, defaults = {}) {
+  const base = normalizeMetadata({ ...defaults, source });
+  const emit = (level, message, details, metadata = {}) => {
+    const overrides = compactMetadata(normalizeMetadata(metadata));
+    return writeLog(level, details === undefined ? [message] : [message, details], {
+      ...base,
+      ...overrides
+    });
+  };
+  return Object.freeze({
+    log: emit,
+    debug: (message, details, metadata) => emit("debug", message, details, metadata),
+    info: (message, details, metadata) => emit("info", message, details, metadata),
+    warn: (message, details, metadata) => emit("warn", message, details, metadata),
+    error: (message, details, metadata) => emit("error", message, details, metadata),
+    child: (childSource, childDefaults = {}) => createLogger(
+      [base.source, normalizeLabel(childSource, "")].filter(Boolean).join(":"),
+      { ...base, ...childDefaults }
+    ),
+    run: (metadata, callback) => withLogContext({ ...base, ...metadata }, callback)
+  });
+}
+
+function withLogContext(metadata, callback) {
+  const parent = contextStorage.getStore() || {};
+  return contextStorage.run({ ...parent, ...compactMetadata(normalizeMetadata(metadata)) }, callback);
+}
+
+function writeLog(level, args, metadata = {}) {
   const safeLevel = LEVELS.has(level) ? level : "info";
   const entry = normalizeArgs(args);
+  const normalized = normalizeMetadata(metadata);
   const createdAt = Date.now();
   try {
-    db.prepare(
+    const result = db.prepare(
-      "INSERT INTO logs (level, message, details, created_at) VALUES (?, ?, ?, ?)"
+      "INSERT INTO logs (level, message, details, source, category, event, request_id, created_at) " +
-    ).run(safeLevel, entry.message, entry.details, createdAt);
+        "VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
+    ).run(
+      safeLevel,
+      truncate(entry.message, MAX_MESSAGE_LENGTH),
+      truncate(entry.details, MAX_DETAILS_LENGTH),
+      normalized.source || "core",
+      normalized.category || "general",
+      normalized.event || null,
+      normalized.requestId || null,
+      createdAt
+    );
+    const stored = {
+      id: Number(result.lastInsertRowid),
+      level: safeLevel,
+      message: truncate(entry.message, MAX_MESSAGE_LENGTH),
+      details: truncate(entry.details, MAX_DETAILS_LENGTH),
+      source: normalized.source || "core",
+      category: normalized.category || "general",
+      event: normalized.event || null,
+      request_id: normalized.requestId || null,
+      created_at: createdAt
+    };
+    publishLogEvent(stored);
+    return stored;
   } catch {
-    // Avoid throwing from logger.
+    return null;
   }
 }
 
 function listLogs(options = {}) {
-  const limit =
+  const query = buildLogQuery(options);
-    Number.isFinite(options.limit) && options.limit !== null
+  let sql =
-      ? Math.max(1, options.limit)
+    "SELECT id, level, message, details, source, category, event, request_id, created_at FROM logs";
-      : null;
+  if (query.clauses.length) sql += ` WHERE ${query.clauses.join(" AND ")}`;
-  const sinceMs =
+  sql += " ORDER BY created_at DESC, id DESC";
-    Number.isFinite(options.sinceMs) && options.sinceMs > 0
+  if (query.limit) {
-      ? options.sinceMs
+    sql += " LIMIT ?";
-      : null;
+    query.params.push(query.limit);
-  const levels = Array.isArray(options.levels)
+  }
-    ? options.levels.filter((level) => LEVELS.has(level))
+  return db.prepare(sql).all(...query.params);
-    : [];
+}
 
+function summarizeLogs(options = {}) {
+  const query = buildLogQuery({ ...options, limit: null });
+  let sql = "SELECT level, COUNT(*) AS count FROM logs";
+  if (query.clauses.length) sql += ` WHERE ${query.clauses.join(" AND ")}`;
+  sql += " GROUP BY level";
+  const levels = { error: 0, warn: 0, info: 0, debug: 0 };
+  let total = 0;
+  for (const row of db.prepare(sql).all(...query.params)) {
+    if (levels[row.level] !== undefined) levels[row.level] = Number(row.count) || 0;
+    total += Number(row.count) || 0;
+  }
+  return { total, levels };
+}
+
+function listLogFacets(options = {}) {
+  const sinceMs = Number.isFinite(options.sinceMs) && options.sinceMs > 0 ? options.sinceMs : null;
+  const where = sinceMs ? " WHERE created_at >= ?" : "";
+  const params = sinceMs ? [sinceMs] : [];
+  const facetRows = (column) => {
+    const connector = sinceMs ? " AND" : " WHERE";
+    return db.prepare(
+      `SELECT ${column} AS value, COUNT(*) AS count FROM logs${where}${connector} ${column} IS NOT NULL AND ${column} != '' ` +
+        `GROUP BY ${column} ORDER BY count DESC, ${column} ASC`
+    ).all(...params).map((row) => ({ value: row.value, count: Number(row.count) || 0 }));
+  };
+  return { sources: facetRows("source"), categories: facetRows("category") };
+}
+
+function cleanupLogs(options = {}) {
+  const maxAgeDays = clampNumber(options.maxAgeDays, 1, 3650, DEFAULT_MAX_AGE_DAYS);
+  const maxEntries = clampNumber(options.maxEntries, 1000, 1000000, DEFAULT_MAX_ENTRIES);
+  const cutoff = Date.now() - maxAgeDays * 24 * 60 * 60 * 1000;
+  const expired = db.prepare("DELETE FROM logs WHERE created_at < ?").run(cutoff).changes;
+  const overflow = db.prepare(
+    "DELETE FROM logs WHERE id NOT IN (SELECT id FROM logs ORDER BY created_at DESC, id DESC LIMIT ?)"
+  ).run(maxEntries).changes;
+  return { removed: expired + overflow, expired, overflow, maxAgeDays, maxEntries };
+}
+
+function hookConsole() {
+  if (consoleHooked) return;
+  consoleHooked = true;
+
+  const original = {
+    debug: console.debug || console.log,
+    log: console.log,
+    info: console.info || console.log,
+    warn: console.warn || console.log,
+    error: console.error || console.log
+  };
+  const capture = (level, method, args) => {
+    const context = contextStorage.getStore() || {};
+    writeLog(level, args, {
+      source: context.source || "console",
+      category: context.category || "runtime",
+      event: context.event,
+      requestId: context.requestId
+    });
+    original[method].apply(console, args);
+  };
+  console.debug = (...args) => capture("debug", "debug", args);
+  console.log = (...args) => capture("info", "log", args);
+  console.info = (...args) => capture("info", "info", args);
+  console.warn = (...args) => capture("warn", "warn", args);
+  console.error = (...args) => capture("error", "error", args);
+}
+
+function buildLogQuery(options = {}) {
+  const limit = Number.isFinite(options.limit) && options.limit !== null
+    ? Math.max(1, Math.floor(options.limit))
+    : null;
+  const sinceMs = Number.isFinite(options.sinceMs) && options.sinceMs > 0 ? options.sinceMs : null;
+  const levels = normalizeList(options.levels, LEVELS);
+  const sources = normalizeList(options.sources);
+  const categories = normalizeList(options.categories);
+  const search = String(options.search || "").trim().slice(0, 200);
   const clauses = [];
   const params = [];
   if (sinceMs) {
     clauses.push("created_at >= ?");
     params.push(sinceMs);
   }
-  if (levels.length) {
+  appendInFilter(clauses, params, "level", levels);
-    clauses.push(`level IN (${levels.map(() => "?").join(",")})`);
+  appendInFilter(clauses, params, "source", sources);
-    params.push(...levels);
+  appendInFilter(clauses, params, "category", categories);
+  if (search) {
+    const needle = `%${escapeLike(search)}%`;
+    clauses.push(
+      "(message LIKE ? ESCAPE '\\' OR details LIKE ? ESCAPE '\\' OR source LIKE ? ESCAPE '\\' " +
+        "OR category LIKE ? ESCAPE '\\' OR event LIKE ? ESCAPE '\\' OR request_id LIKE ? ESCAPE '\\')"
+    );
+    params.push(needle, needle, needle, needle, needle, needle);
   }
-
+  return { clauses, params, limit };
-  let query =
-    "SELECT id, level, message, details, created_at FROM logs";
-  if (clauses.length) {
-    query += ` WHERE ${clauses.join(" AND ")}`;
-  }
-  query += " ORDER BY created_at DESC";
-  if (limit) {
-    query += " LIMIT ?";
-    params.push(limit);
-  }
-
-  return db.prepare(query).all(...params);
 }
 
-function hookConsole() {
+function appendInFilter(clauses, params, column, values) {
-  if (consoleHooked) {
+  if (!values.length) return;
-    return;
+  clauses.push(`${column} IN (${values.map(() => "?").join(",")})`);
-  }
+  params.push(...values);
-  consoleHooked = true;
+}
 
-  const original = {
+function normalizeList(values, allowed = null) {
-    log: console.log,
+  const list = Array.isArray(values) ? values : values ? [values] : [];
-    info: console.info || console.log,
+  return [...new Set(list.map((value) => normalizeLabel(value, "")).filter((value) =>
-    warn: console.warn || console.log,
+    value && (!allowed || allowed.has(value))
-    error: console.error || console.log
+  ))];
-  };
+}
 
-  console.log = (...args) => {
+function normalizeMetadata(metadata = {}) {
-    log("info", ...args);
+  return {
-    original.log.apply(console, args);
+    source: normalizeLabel(metadata.source, ""),
-  };
+    category: normalizeLabel(metadata.category, ""),
-  console.info = (...args) => {
+    event: normalizeLabel(metadata.event, ""),
-    log("info", ...args);
+    requestId: normalizeLabel(metadata.requestId || metadata.request_id, "")
-    original.info.apply(console, args);
-  };
-  console.warn = (...args) => {
-    log("warn", ...args);
-    original.warn.apply(console, args);
-  };
-  console.error = (...args) => {
-    log("error", ...args);
-    original.error.apply(console, args);
   };
 }
 
+function compactMetadata(metadata) {
+  return Object.fromEntries(Object.entries(metadata).filter(([, value]) => value));
+}
+
+function normalizeLabel(value, fallback) {
+  const normalized = String(value || "").trim().toLowerCase().replace(/[^a-z0-9_.:-]+/g, "-").replace(/^-+|-+$/g, "");
+  return normalized.slice(0, 80) || fallback;
+}
+
 function normalizeArgs(args) {
-  if (!args || args.length === 0) {
+  if (!args || args.length === 0) return { message: "Log entry", details: "" };
-    return { message: "Log entry", details: "" };
-  }
   let message = "";
   const detailParts = [];
-
   const first = args[0];
   if (first instanceof Error) {
-    message = first.message || "Error";
+    message = redactText(first.message || "Error");
-    detailParts.push(first.stack || String(first));
+    detailParts.push(redactText(first.stack || String(first)));
   } else {
     message = formatArg(first);
   }
-
   for (const arg of args.slice(1)) {
     if (arg instanceof Error) {
-      detailParts.push(arg.stack || arg.message || String(arg));
+      detailParts.push(redactText(arg.stack || arg.message || String(arg)));
-      if (!message) {
+      if (!message) message = redactText(arg.message || "Error");
-        message = arg.message || "Error";
-      }
     } else {
       detailParts.push(formatArg(arg));
     }
   }
-
-  if (!message) {
-    message = "Log entry";
-  }
-
   return {
-    message,
+    message: message || "Log entry",
     details: detailParts.filter(Boolean).join("\n")
   };
 }
 
 function formatArg(value) {
-  if (typeof value === "string") {
+  if (typeof value === "string") return redactText(value);
-    return value;
+  if (value instanceof Error) return redactText(value.stack || value.message || String(value));
-  }
+  return redactText(util.inspect(redactValue(value), {
-  if (value instanceof Error) {
+    depth: 6,
-    return value.stack || value.message || String(value);
+    maxArrayLength: 100,
-  }
+    maxStringLength: 4000,
-  return util.inspect(value, {
-    depth: 4,
-    maxArrayLength: 50,
     breakLength: 120
-  });
+  }));
+}
+
+function redactValue(value, seen = new WeakSet()) {
+  if (value === null || value === undefined) return value;
+  if (typeof value === "string") return redactText(value);
+  if (typeof value !== "object") return value;
+  if (value instanceof Error) return redactText(value.stack || value.message || String(value));
+  if (seen.has(value)) return "[Circular]";
+  seen.add(value);
+  if (Array.isArray(value)) return value.map((item) => redactValue(item, seen));
+  const output = {};
+  for (const [key, item] of Object.entries(value)) {
+    output[key] = isSensitiveKey(key) ? "[REDACTED]" : redactValue(item, seen);
+  }
+  return output;
+}
+
+function isSensitiveKey(key) {
+  return /(?:^|[_-])(authorization|cookie|password|passwd|secret|token|api[_-]?key|access[_-]?token|refresh[_-]?token|client[_-]?secret|private[_-]?key)(?:$|[_-])/i.test(String(key));
+}
+
+function redactText(value) {
+  return String(value || "")
+    .replace(/\b(Bearer|Basic)\s+[A-Za-z0-9._~+/=-]+/gi, "$1 [REDACTED]")
+    .replace(/([?&](?:token|key|secret|password|authorization)=)[^&#\s]+/gi, "$1[REDACTED]")
+    .replace(/\b((?:api[_-]?key|access[_-]?token|refresh[_-]?token|client[_-]?secret|password|passwd|secret|authorization)\s*[:=]\s*)[^\s,;}]+/gi, "$1[REDACTED]");
+}
+
+function truncate(value, maxLength) {
+  const text = String(value || "");
+  if (text.length <= maxLength) return text;
+  return `${text.slice(0, maxLength)}\n[truncated]`;
+}
+
+function escapeLike(value) {
+  return String(value).replace(/[\\%_]/g, (character) => `\\${character}`);
+}
+
+function clampNumber(value, minimum, maximum, fallback) {
+  const parsed = Number(value);
+  if (!Number.isFinite(parsed)) return fallback;
+  return Math.min(maximum, Math.max(minimum, Math.floor(parsed)));
+}
+
+function publishLogEvent(entry) {
+  try {
+    const { publishWebEvent } = require("./web-events");
+    publishWebEvent("log:created", { ...entry, details: truncate(entry.details, 4000) }, { role: "admin" });
+  } catch {
+    // Logging must never fail because live delivery is unavailable.
+  }
 }
 
 module.exports = {
-  log,
+  DEFAULT_MAX_AGE_DAYS,
+  DEFAULT_MAX_ENTRIES,
+  cleanupLogs,
+  createLogger,
+  hookConsole,
+  listLogFacets,
   listLogs,
-  hookConsole
+  log,
+  redactValue,
+  summarizeLogs,
+  withLogContext
 };

src/services/plugins.js

@@ -3,6 +3,9 @@ const fs = require("fs");
 const { spawnSync } = require("child_process");
 const { db } = require("./db");
 const placeholders = require("./placeholders");
+const { createLogger } = require("./logger");
+
+const pluginLifecycleLog = createLogger("core:plugins", { category: "lifecycle" });
 
 const pluginsDir = path.join(__dirname, "..", "..", "plugins");
 const cleanupHandlers = [];
@@ -180,11 +183,15 @@ function setPluginEnabled(id, enabled) {
     Date.now(),
     id
   );
+  pluginLifecycleLog.info(`Plugin ${enabled ? "enabled" : "disabled"}`, { plugin_id: id }, {
+    event: enabled ? "plugin_enabled" : "plugin_disabled"
+  });
 }
 
 function removePlugin(id) {
   db.prepare("DELETE FROM plugins WHERE id = ?").run(id);
   db.prepare("DELETE FROM plugin_settings WHERE plugin_id = ?").run(id);
+  pluginLifecycleLog.warn("Plugin registry entry removed", { plugin_id: id }, { event: "plugin_removed" });
 }
 
 function clearPluginCache(pluginPath) {
@@ -223,30 +230,35 @@ function loadEnabled({
       continue;
     }
     clearPluginCache(plugin.dir);
+    const pluginLog = createLogger(`plugin:${plugin.id}`, { category: "plugin" });
     try {
-      const mod = require(mainPath);
+      pluginLog.run({ event: "plugin_load" }, () => {
-      if (mod && typeof mod.init === "function") {
+        const mod = require(mainPath);
-        const cleanup = mod.init({
+        if (mod && typeof mod.init === "function") {
-          app,
+          const cleanup = mod.init({
-          discordClient,
+            app,
-          twitchClient,
+            discordClient,
-          youtubeClient,
+            twitchClient,
-          settings,
+            youtubeClient,
-          web,
+            settings,
-          webhooks,
+            web,
-          db,
+            webhooks,
-          plugin,
+            db,
-          commandRouter,
+            plugin,
-          placeholders
+            commandRouter,
-        });
+            placeholders,
-        if (typeof cleanup === "function") {
+            logger: pluginLog
-          cleanupHandlers.push({ id: plugin.id, cleanup });
+          });
-        } else if (cleanup && typeof cleanup.stop === "function") {
+          if (typeof cleanup === "function") {
-          cleanupHandlers.push({ id: plugin.id, cleanup: () => cleanup.stop() });
+            cleanupHandlers.push({ id: plugin.id, cleanup });
+          } else if (cleanup && typeof cleanup.stop === "function") {
+            cleanupHandlers.push({ id: plugin.id, cleanup: () => cleanup.stop() });
+          }
         }
-      }
+      });
+      pluginLog.info("Plugin loaded", { version: plugin.version }, { event: "plugin_loaded" });
     } catch (error) {
-      console.error(`Plugin ${plugin.id} failed to load`, error);
+      pluginLog.error("Plugin failed to load", error, { event: "plugin_load_failed" });
     }
   }
 }
@@ -256,8 +268,10 @@ async function stopPlugins() {
   for (const handler of handlers) {
     try {
       await handler.cleanup();
+      pluginLifecycleLog.info("Plugin stopped", { plugin_id: handler.id }, { event: "plugin_stopped" });
     } catch (error) {
-      console.error(`Plugin ${handler.id} failed to stop`, error);
+      createLogger(`plugin:${handler.id}`, { category: "lifecycle" })
+        .error("Plugin failed to stop", error, { event: "plugin_stop_failed" });
     }
   }
 }
@@ -284,6 +298,7 @@ function installFromGit(url, targetFolder) {
   if (result.status !== 0) {
     throw new Error(result.stderr || "Git clone failed.");
   }
+  pluginLifecycleLog.info("Plugin cloned from repository", { plugin_id: folderName }, { event: "plugin_installed" });
   return targetPath;
 }
 
@@ -295,6 +310,7 @@ function updatePluginFromGit(pluginPath) {
   if (result.status !== 0) {
     throw new Error(result.stderr || "Git pull failed.");
   }
+  pluginLifecycleLog.info("Plugin repository updated", { plugin_id: path.basename(pluginPath) }, { event: "plugin_updated" });
   return result.stdout;
 }
 
@@ -337,6 +353,7 @@ function createLocalPlugin({ id, name, description }) {
   }
 };\n`;
   fs.writeFileSync(mainPath, starter, "utf8");
+  pluginLifecycleLog.info("Local plugin created", { plugin_id: safeId }, { event: "plugin_created" });
   return pluginDir;
 }
 

src/services/production-diagnostics.js

@@ -4,7 +4,7 @@ const path = require("path");
 const { performance } = require("perf_hooks");
 const { db } = require("./db");
 const { dependencyIssues } = require("./dependency-manager");
-const { listLogs, log } = require("./logger");
+const { createLogger, listLogs } = require("./logger");
 const { getPlugins, scanPluginDirectories } = require("./plugins");
 const { readRecoveryMarker } = require("./recovery-mode");
 const { getSetting, setSetting } = require("./settings");
@@ -15,6 +15,7 @@ const packageJson = require(path.join(repoRoot, "package.json"));
 const TOKEN_PREFIX = "lumi_diag_";
 const MAX_REQUESTS_PER_MINUTE = 20;
 const requestWindows = new Map();
+const diagnosticsLog = createLogger("core:diagnostics", { category: "security" });
 const CHECKS = Object.freeze({
   system_health: "Runtime, database, dependency, disk-space, and recovery health.",
   update_state: "Latest local update state, recovery marker, and snapshot summary.",
@@ -39,7 +40,7 @@ function issueDiagnosticsAccessKey() {
   setSetting("production_diagnostics_key_prefix", `${key.slice(0, TOKEN_PREFIX.length + 8)}…`);
   setSetting("production_diagnostics_key_created_at", new Date().toISOString());
   setSetting("production_diagnostics_enabled", true);
-  log("warn", "Production diagnostics access key rotated", { key_prefix: `${key.slice(0, TOKEN_PREFIX.length + 8)}…` });
+  diagnosticsLog.warn("Production diagnostics access key rotated", { key_prefix: `${key.slice(0, TOKEN_PREFIX.length + 8)}…` }, { event: "access_rotated" });
   return key;
 }
 
@@ -49,7 +50,7 @@ function revokeDiagnosticsAccess() {
   setSetting("production_diagnostics_key_prefix", "");
   setSetting("production_diagnostics_key_created_at", null);
   requestWindows.clear();
-  log("warn", "Production diagnostics access revoked");
+  diagnosticsLog.warn("Production diagnostics access revoked", null, { event: "access_revoked" });
 }
 
 function authenticateDiagnosticsRequest(req, now = Date.now()) {
@@ -86,14 +87,14 @@ function runDiagnosticCheck(checkId) {
 }
 
 function auditDiagnosticRequest(values = {}) {
-  log(values.ok === false ? "warn" : "info", "Production diagnostics request", {
+  diagnosticsLog.log(values.ok === false ? "warn" : "info", "Production diagnostics request", {
     request_id: values.request_id,
     check: values.check,
     ok: values.ok !== false,
     key_fingerprint: values.fingerprint || null,
     reason: values.reason || null,
     duration_ms: values.duration_ms || null
-  });
+  }, { event: "diagnostic_request", requestId: values.request_id });
 }
 
 function recentDiagnosticAudit(limit = 30) {

src/services/settings.js

@@ -69,6 +69,8 @@ function ensureDefaults() {
     auto_update_enabled: envBoolean("AUTO_UPDATE_ENABLED", false),
     auto_update_interval_minutes: envNumber("AUTO_UPDATE_INTERVAL_MINUTES", 60),
     production_diagnostics_enabled: false,
+    log_retention_days: 30,
+    log_retention_max_entries: 100000,
     git_remote: envString("GIT_REMOTE", "origin"),
     git_branch: envString("GIT_BRANCH", "main"),
     bot_avatar_url: null,

src/services/twitch.js

@@ -2,6 +2,9 @@ const tmi = require("tmi.js");
 const { getSetting } = require("./settings");
 const { incrementMessages } = require("./stats");
 const { ensureUserForIdentity } = require("./users");
+const { createLogger } = require("./logger");
+
+const twitchLog = createLogger("platform:twitch", { category: "integration" });
 
 let client = null;
 
@@ -32,7 +35,7 @@ async function startTwitchBot({ commandRouter } = {}) {
   });
 
   client.on("connected", (address, port) => {
-    console.log(`Twitch bot connected to ${address}:${port}`);
+    twitchLog.info("Twitch bot connected", { address, port, channels }, { event: "platform_ready" });
   });
 
   client.on("message", async (channel, tags, message, self) => {
@@ -66,7 +69,7 @@ async function startTwitchBot({ commandRouter } = {}) {
           try {
             await client.say(channel, content);
           } catch (error) {
-            console.error("Twitch command reply failed", error);
+            twitchLog.error("Twitch command reply failed", error, { event: "reply_failed" });
           }
         }
       });

src/services/webhooks.js

@@ -1,6 +1,8 @@
 const crypto = require("crypto");
 const express = require("express");
-const { log } = require("./logger");
+const { createLogger } = require("./logger");
+
+const webhookLog = createLogger("core:webhooks", { category: "integration" });
 
 function createWebhookService({ limit = "256kb" } = {}) {
   const endpoints = new Map();
@@ -60,14 +62,14 @@ function createWebhookService({ limit = "256kb" } = {}) {
       }
       return sendHandlerResult(res, await endpoint.handler(context));
     } catch (error) {
-      log("error", "Webhook handler failed", {
+      webhookLog.error("Webhook handler failed", {
         pluginId: endpoint.pluginId,
         endpointId: endpoint.endpointId,
         namespace,
         slug,
         message: error?.message || String(error),
         stack: error?.stack || ""
-      });
+      }, { event: "inbound_webhook_failed" });
       if (!res.headersSent) {
         return res.status(500).json({ error: "Webhook processing failed." });
       }
@@ -76,11 +78,11 @@ function createWebhookService({ limit = "256kb" } = {}) {
   });
   router.use((error, req, res, _next) => {
     const status = error?.type === "entity.too.large" ? 413 : 400;
-    log("warn", "Webhook request rejected", {
+    webhookLog.warn("Webhook request rejected", {
       path: req.path,
       status,
       message: error?.message || String(error)
-    });
+    }, { event: "inbound_webhook_rejected" });
     res.status(status).json({
       error: status === 413 ? "Webhook payload is too large." : "Invalid webhook request."
     });
@@ -124,11 +126,11 @@ function createWebhookService({ limit = "256kb" } = {}) {
       endpointKeysByPlugin.set(safePluginId, new Map());
     }
     endpointKeysByPlugin.get(safePluginId).set(safeEndpointId, key);
-    log("info", "Webhook endpoint registered", {
+    webhookLog.info("Webhook endpoint registered", {
       pluginId: safePluginId,
       endpointId: safeEndpointId,
       path: `/webhooks/${key}`
-    });
+    }, { event: "webhook_registered" });
     return { namespace: safeNamespace, slug: safeSlug, path: `/webhooks/${key}` };
   }
 
@@ -143,7 +145,7 @@ function createWebhookService({ limit = "256kb" } = {}) {
     if (!pluginEndpoints.size) {
       endpointKeysByPlugin.delete((pluginId || "").toString());
     }
-    log("debug", "Webhook endpoint unregistered", { pluginId, endpointId });
+    webhookLog.debug("Webhook endpoint unregistered", { pluginId, endpointId }, { event: "webhook_unregistered" });
     return true;
   }
 
@@ -229,12 +231,12 @@ async function sendWebhook({
       };
       if (response.ok || attempt === attempts) {
         if (!response.ok) {
-          log("warn", "Outbound webhook returned an error", {
+          webhookLog.warn("Outbound webhook returned an error", {
             pluginId: pluginId || null,
             url: redactUrl(url),
             status: response.status,
             attempt
-          });
+          }, { event: "outbound_webhook_error" });
         }
         return result;
       }
@@ -242,12 +244,12 @@ async function sendWebhook({
       lastError = error;
       lastDurationMs = Date.now() - startedAt;
       if (attempt === attempts) {
-        log("error", "Outbound webhook failed", {
+        webhookLog.error("Outbound webhook failed", {
           pluginId: pluginId || null,
           url: redactUrl(url),
           attempt,
           message: error?.message || String(error)
-        });
+        }, { event: "outbound_webhook_failed" });
       }
     } finally {
       clearTimeout(timer);

src/services/youtube.js

@@ -1,6 +1,9 @@
 const { getSetting, setSetting } = require("./settings");
 const { incrementMessages } = require("./stats");
 const { ensureUserForIdentity } = require("./users");
+const { createLogger } = require("./logger");
+
+const youtubeLog = createLogger("platform:youtube", { category: "integration" });
 
 let client = null;
 let pollTimer = null;
@@ -43,7 +46,7 @@ async function startYouTubeBot({ commandRouter } = {}) {
   try {
     await hydrateBotChannel(state);
   } catch (error) {
-    console.error("YouTube bot failed to load channel details", error);
+    youtubeLog.error("YouTube bot failed to load channel details", error, { event: "channel_load_failed" });
   }
 
   schedulePoll(state, 1000);
@@ -75,7 +78,7 @@ function schedulePoll(state, delayMs) {
   pollTimer = setTimeout(() => {
     pollTimer = null;
     pollLiveChat(state).catch((error) => {
-      console.error("YouTube chat poll failed", error);
+      youtubeLog.error("YouTube chat poll failed", error, { event: "poll_failed" });
       schedulePoll(state, 10000);
     });
   }, delayMs);
@@ -154,7 +157,7 @@ async function handleChatItem(state, liveChatId, item) {
       try {
         await sendChatMessage(state, liveChatId, content);
       } catch (error) {
-        console.error("YouTube command reply failed", error);
+        youtubeLog.error("YouTube command reply failed", error, { event: "reply_failed" });
       }
     }
   });

src/web/public/app.js

@@ -1001,16 +1001,18 @@
 
   const logList = document.querySelector("[data-log-list]");
   if (logList) {
-    const entries = Array.from(logList.querySelectorAll("[data-log-entry]"));
     const searchInput = document.querySelector("[data-log-search]");
     const levelSelect = document.querySelector("[data-log-level]");
+    const sourceSelect = document.querySelector("[data-log-source]");
+    const categorySelect = document.querySelector("[data-log-category]");
     const rangeSelect = document.querySelector("[data-log-range]");
     const limitSelect = document.querySelector("[data-log-limit]");
+    const liveStatus = document.querySelector("[data-log-live-status] .status-indicator");
 
     const applyLogFilters = () => {
       const term = (searchInput?.value || "").trim().toLowerCase();
 
-      entries.forEach((entry) => {
+      Array.from(logList.querySelectorAll("[data-log-entry]")).forEach((entry) => {
         const haystack = (entry.dataset.search || entry.textContent || "")
           .toLowerCase()
           .trim();
@@ -1025,19 +1027,97 @@
     applyLogFilters();
 
     const reloadLogView = () => {
-      const url = new URL(window.location.href);
+      document.querySelector("[data-log-filter-form]")?.requestSubmit();
-      const rangeValue = rangeSelect?.value || "all";
-      const levelValue = levelSelect?.value || "all";
-      const limitValue = limitSelect?.value || "50";
-      url.searchParams.set("range", rangeValue);
-      url.searchParams.set("level", levelValue);
-      url.searchParams.set("limit", limitValue);
-      window.location.assign(url.toString());
     };
 
     levelSelect?.addEventListener("change", reloadLogView);
+    sourceSelect?.addEventListener("change", reloadLogView);
+    categorySelect?.addEventListener("change", reloadLogView);
     rangeSelect?.addEventListener("change", reloadLogView);
     limitSelect?.addEventListener("change", reloadLogView);
+
+    const matchesLiveFilters = (entry) => {
+      if (levelSelect?.value !== "all" && entry.level !== levelSelect?.value) return false;
+      if (sourceSelect?.value !== "all" && entry.source !== sourceSelect?.value) return false;
+      if (categorySelect?.value !== "all" && entry.category !== categorySelect?.value) return false;
+      const range = rangeSelect?.value || "all";
+      if (range !== "all" && Number(entry.created_at) < Date.now() - Number(range)) return false;
+      const term = (searchInput?.value || "").trim().toLowerCase();
+      const haystack = [entry.message, entry.details, entry.source, entry.category, entry.event, entry.request_id]
+        .filter(Boolean).join(" ").toLowerCase();
+      return !term || haystack.includes(term);
+    };
+
+    const createLiveLogEntry = (entry) => {
+      const details = document.createElement("details");
+      details.className = `log-entry level-${entry.level}`;
+      details.dataset.logEntry = "";
+      details.dataset.level = entry.level;
+      details.dataset.source = entry.source || "core";
+      details.dataset.category = entry.category || "general";
+      details.dataset.timestamp = String(entry.created_at || Date.now());
+      details.dataset.search = [entry.message, entry.details, entry.source, entry.category, entry.event, entry.request_id]
+        .filter(Boolean).join(" ").toLowerCase();
+      const summary = document.createElement("summary");
+      const marker = document.createElement("span");
+      marker.className = "log-marker";
+      marker.setAttribute("aria-hidden", "true");
+      const message = document.createElement("span");
+      message.className = "log-message";
+      message.textContent = entry.message || "Log entry";
+      const scope = document.createElement("span");
+      scope.className = "log-scope-pill";
+      scope.textContent = entry.source || "core";
+      const level = document.createElement("span");
+      level.className = "log-level-pill";
+      level.textContent = entry.level || "info";
+      const time = document.createElement("span");
+      time.className = "log-time";
+      time.textContent = new Date(entry.created_at || Date.now()).toLocaleString();
+      summary.append(marker, message, scope, level, time);
+      const meta = document.createElement("div");
+      meta.className = "log-entry-meta";
+      const values = [
+        ["Activity", entry.category || "general"],
+        ["Event", entry.event],
+        ["Request ID", entry.request_id]
+      ];
+      values.forEach(([label, value]) => {
+        if (!value) return;
+        const item = document.createElement("span");
+        const strong = document.createElement("strong");
+        strong.textContent = `${label}:`;
+        item.append(strong, ` ${value}`);
+        meta.append(item);
+      });
+      const body = entry.details ? document.createElement("pre") : document.createElement("div");
+      body.className = entry.details ? "log-details" : "log-details empty";
+      body.textContent = entry.details || "No additional details.";
+      details.append(summary, meta, body);
+      return details;
+    };
+
+    window.addEventListener("lumi:log-created", (event) => {
+      const entry = event.detail || {};
+      if (!entry.id || !matchesLiveFilters(entry)) return;
+      logList.querySelector("[data-log-empty]")?.remove();
+      logList.prepend(createLiveLogEntry(entry));
+      const limit = Number(logList.dataset.logLimitValue || 50);
+      const entries = Array.from(logList.querySelectorAll("[data-log-entry]"));
+      entries.slice(limit).forEach((item) => item.remove());
+      const total = document.querySelector("[data-log-total]");
+      if (total) total.textContent = String(Number(total.textContent || 0) + 1);
+      const levelCount = document.querySelector(`[data-log-count="${entry.level}"]`);
+      if (levelCount) levelCount.textContent = String(Number(levelCount.textContent || 0) + 1);
+      if (liveStatus) liveStatus.textContent = "Live · newest entry added";
+    });
+    window.addEventListener("lumi:event-status", (event) => {
+      if (!liveStatus) return;
+      const connected = event.detail?.status === "connected";
+      liveStatus.textContent = connected ? "Live updates connected" : "Live updates reconnecting";
+      liveStatus.classList.toggle("status-success", connected);
+      liveStatus.classList.toggle("status-warning", !connected);
+    });
   }
 
   const logModal = document.querySelector("[data-log-modal]");

src/web/public/lumi-interactions.js

@@ -188,12 +188,19 @@
     stream.addEventListener("server:warning", (event) => showEventNotice(readEvent(event), "warning"));
     stream.addEventListener("server:status", (event) => {
       const data = readEvent(event);
-      if (data.status === "connected") document.body.dataset.eventStream = "connected";
+      if (data.status === "connected") {
+        document.body.dataset.eventStream = "connected";
+        window.dispatchEvent(new CustomEvent("lumi:event-status", { detail: { status: "connected" } }));
+      }
     });
     stream.addEventListener("ai:model_status", (event) => showEventNotice(readEvent(event), "danger"));
     stream.addEventListener("data:new_available", (event) => showRefreshPrompt(readEvent(event)));
+    stream.addEventListener("log:created", (event) => {
+      window.dispatchEvent(new CustomEvent("lumi:log-created", { detail: readEvent(event) }));
+    });
     stream.onerror = () => {
       document.body.dataset.eventStream = "disconnected";
+      window.dispatchEvent(new CustomEvent("lumi:event-status", { detail: { status: "disconnected" } }));
     };
   }
 

src/web/public/styles.css

@@ -1039,6 +1039,46 @@ body {
   border: 1px solid var(--border);
 }
 
+.log-scope-pill {
+  max-width: 15rem;
+  padding: 4px 8px;
+  border: 1px solid color-mix(in srgb, var(--sea) 45%, var(--border));
+  border-radius: 999px;
+  background: color-mix(in srgb, var(--sea) 12%, var(--surface-3));
+  color: var(--ink-soft);
+  font-size: 0.75rem;
+  font-weight: 700;
+  overflow: hidden;
+  text-overflow: ellipsis;
+  white-space: nowrap;
+}
+
+.log-entry-meta {
+  display: flex;
+  flex-wrap: wrap;
+  gap: 8px 18px;
+  margin: 10px 2px 0;
+  color: var(--ink-soft);
+  font-size: 0.82rem;
+}
+
+.log-live-status {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  gap: 12px;
+  min-height: 2.75rem;
+  margin-top: 14px;
+}
+
+.log-summary-grid {
+  margin: 18px 0;
+}
+
+.log-storage-settings {
+  margin-top: 16px;
+}
+
 .log-time {
   font-size: 0.85rem;
   color: var(--ink-soft);
@@ -1063,6 +1103,22 @@ body {
   color: var(--ink-soft);
 }
 
+@media (max-width: 760px) {
+  .log-entry summary {
+    align-items: flex-start;
+    flex-wrap: wrap;
+  }
+
+  .log-message {
+    flex-basis: calc(100% - 22px);
+  }
+
+  .log-time {
+    width: 100%;
+    padding-left: 20px;
+  }
+}
+
 .identity-list {
   list-style: none;
   padding: 0;

src/web/server.js

@@ -51,7 +51,17 @@ const {
   getLeaderboardSections,
   getTopCommandOptions
 } = require("../services/top");
-const { log, listLogs } = require("../services/logger");
+const {
+  DEFAULT_MAX_AGE_DAYS,
+  DEFAULT_MAX_ENTRIES,
+  cleanupLogs,
+  createLogger,
+  listLogFacets,
+  listLogs,
+  log,
+  summarizeLogs,
+  withLogContext
+} = require("../services/logger");
 const { createWebhookService } = require("../services/webhooks");
 const {
   getPlatformStatus,
@@ -1344,6 +1354,7 @@ const DASHBOARD_SCOPES = {
   "7d": 7 * 24 * 60 * 60 * 1000
 };
 const memorySamples = [];
+const webLog = createLogger("core:web", { category: "http" });
 
 function normalizeLogLevel(value) {
   const normalized = (value || "").toString().trim().toLowerCase();
@@ -1358,6 +1369,14 @@ function parseLogLevels(value) {
   return raw.map(normalizeLogLevel).filter(Boolean);
 }
 
+function normalizeLogFacet(value) {
+  return String(value || "").trim().toLowerCase().replace(/[^a-z0-9_.:-]+/g, "-").slice(0, 80);
+}
+
+function parseLogSearch(value) {
+  return String(value || "").trim().slice(0, 200);
+}
+
 function parseLogRange(value) {
   if (value === undefined || value === null || value === "") {
     return { rangeMs: DEFAULT_LOG_RANGE_MS, rangeValue: `${DEFAULT_LOG_RANGE_MS}` };
@@ -2977,6 +2996,44 @@ function createWebServer({ loadPlugins, discordClient }) {
   );
   app.use(express.urlencoded({ extended: false }));
   app.use(express.json({ limit: "1mb" }));
+  app.use((req, res, next) => {
+    const requestId = crypto.randomUUID();
+    const startedAt = Date.now();
+    const method = String(req.method || "GET").toUpperCase();
+    const requestPath = String(req.path || "/");
+    const pluginMatch = requestPath.match(/^\/plugins\/([^/]+)/);
+    const source = pluginMatch ? `plugin:${normalizeLogFacet(pluginMatch[1])}` : "core:web";
+    res.setHeader("X-Request-Id", requestId);
+    return withLogContext({ source, category: "http", requestId }, () => {
+      res.on("finish", () => {
+        if (["/api/events", "/admin/updates/events"].includes(requestPath)) return;
+        const durationMs = Date.now() - startedAt;
+        const status = Number(res.statusCode) || 0;
+        const mutating = !["GET", "HEAD", "OPTIONS"].includes(method);
+        const slow = durationMs >= 2000;
+        if (!mutating && status < 400 && !slow) return;
+        const level = status >= 500 ? "error" : status >= 400 || slow ? "warn" : "info";
+        const category = mutating && status < 400 ? "audit" : "http";
+        const routePath = typeof req.route?.path === "string"
+          ? `${req.baseUrl || ""}${req.route.path}`
+          : requestPath.replace(/\b[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}\b/gi, ":id");
+        webLog.log(level, "HTTP request completed", {
+          method,
+          path: routePath,
+          status,
+          duration_ms: durationMs,
+          user_id: req.session?.user?.id || null,
+          role: req.session?.user?.isAdmin ? "admin" : req.session?.user?.isMod ? "mod" : req.session?.user ? "user" : "anonymous"
+        }, {
+          source,
+          category,
+          event: status >= 400 ? "http_error" : slow ? "slow_request" : "admin_action",
+          requestId
+        });
+      });
+      next();
+    });
+  });
   app.use((_req, res, next) => {
     res.setHeader("X-Content-Type-Options", "nosniff");
     next();
@@ -5833,35 +5890,90 @@ function createWebServer({ loadPlugins, discordClient }) {
     const range = parseLogRange(req.query.range);
     const limit = parseLogLimit(req.query.limit);
     const levelValue = normalizeLogLevel(req.query.level) || "all";
+    const sourceValue = normalizeLogFacet(req.query.source) || "all";
+    const categoryValue = normalizeLogFacet(req.query.category) || "all";
+    const search = parseLogSearch(req.query.q);
     const levels = levelValue === "all" ? [] : [levelValue];
     const sinceMs = range.rangeMs ? Date.now() - range.rangeMs : null;
-    const logs = listLogs({ limit: limit.limit, sinceMs, levels });
+    const query = {
+      limit: limit.limit,
+      sinceMs,
+      levels,
+      sources: sourceValue === "all" ? [] : [sourceValue],
+      categories: categoryValue === "all" ? [] : [categoryValue],
+      search
+    };
+    const logs = listLogs(query);
     res.render("admin-logs", {
       title: "Logs",
       logs,
+      logSummary: summarizeLogs(query),
+      logFacets: listLogFacets({ sinceMs }),
+      logRetention: {
+        maxAgeDays: getSetting("log_retention_days", DEFAULT_MAX_AGE_DAYS),
+        maxEntries: getSetting("log_retention_max_entries", DEFAULT_MAX_ENTRIES)
+      },
       logFilters: {
         range: range.rangeValue,
         level: levelValue,
-        limit: limit.limitValue
+        limit: limit.limitValue,
+        source: sourceValue,
+        category: categoryValue,
+        search
       }
     });
   });
 
+  app.post("/admin/logs/retention", requireRole("admin"), (req, res) => {
+    const cleanup = cleanupLogs({
+      maxAgeDays: req.body.max_age_days,
+      maxEntries: req.body.max_entries
+    });
+    setSetting("log_retention_days", cleanup.maxAgeDays);
+    setSetting("log_retention_max_entries", cleanup.maxEntries);
+    webLog.info("Log retention updated", {
+      max_age_days: cleanup.maxAgeDays,
+      max_entries: cleanup.maxEntries,
+      removed: cleanup.removed,
+      user_id: req.session.user.id
+    }, { category: "audit", event: "log_retention_updated" });
+    setFlash(req, "success", cleanup.removed
+      ? `Log storage updated and ${cleanup.removed} old entr${cleanup.removed === 1 ? "y was" : "ies were"} removed.`
+      : "Log storage settings updated.");
+    res.redirect("/admin/logs");
+  });
+
   app.get("/admin/logs/download", requireRole("admin"), (req, res) => {
     const range = parseLogRange(req.query.range);
     const limit = parseLogLimit(req.query.limit, { allowAll: true });
     const levels = parseLogLevels(req.query.level);
+    const source = normalizeLogFacet(req.query.source);
+    const category = normalizeLogFacet(req.query.category);
+    const search = parseLogSearch(req.query.q);
     const sinceMs = range.rangeMs ? Date.now() - range.rangeMs : null;
-    const logs = listLogs({ limit: limit.limit, sinceMs, levels });
+    const logs = listLogs({
+      limit: limit.limit,
+      sinceMs,
+      levels,
+      sources: source && source !== "all" ? [source] : [],
+      categories: category && category !== "all" ? [category] : [],
+      search
+    });
     const stamp = new Date().toISOString().replace(/[:.]/g, "-");
+    const format = req.query.format === "jsonl" ? "jsonl" : "txt";
     res.setHeader(
       "Content-Disposition",
-      `attachment; filename="lumi-logs-${stamp}.txt"`
+      `attachment; filename="lumi-logs-${stamp}.${format}"`
     );
-    res.setHeader("Content-Type", "text/plain; charset=utf-8");
+    res.setHeader("Content-Type", format === "jsonl" ? "application/x-ndjson; charset=utf-8" : "text/plain; charset=utf-8");
+    if (format === "jsonl") {
+      return res.send(logs.map((entry) => JSON.stringify(entry)).join("\n") + (logs.length ? "\n" : ""));
+    }
     const lines = logs.map((log) => {
       const timestamp = new Date(log.created_at).toISOString();
-      const header = `${timestamp} [${log.level.toUpperCase()}] ${log.message}`;
+      const scope = [log.source, log.category, log.event].filter(Boolean).join(" / ");
+      const request = log.request_id ? ` request=${log.request_id}` : "";
+      const header = `${timestamp} [${log.level.toUpperCase()}] [${scope}]${request} ${log.message}`;
       if (log.details) {
         return `${header}\n${log.details}\n`;
       }

src/web/views/admin-logs.ejs

@@ -1,35 +1,75 @@
 <%- include("partials/layout-top", { title }) %>
-<% const filters = logFilters || { range: '86400000', level: 'all', limit: '50' }; %>
+<%
+  const filters = logFilters || { range: '86400000', level: 'all', limit: '50', source: 'all', category: 'all', search: '' };
+  const summary = logSummary || { total: 0, levels: { error: 0, warn: 0, info: 0, debug: 0 } };
+  const facets = logFacets || { sources: [], categories: [] };
+  const retention = logRetention || { maxAgeDays: 30, maxEntries: 100000 };
+%>
 <section class="card">
-  <div class="section-header">
+  <%- include("partials/page-header", {
-    <div>
+    eyebrow: "Administration",
-      <h1>Logs</h1>
+    pageTitle: "System logs",
-      <p class="command-subtitle">Core system logs with severity, timestamps, and details.</p>
+    description: "Review activity from Lumi Core, plugins, commands, integrations, and administrator actions. Sensitive values are removed before entries are stored."
-    </div>
+  }) %>
-    <div class="log-controls">
+
-      <label>
+  <div class="dashboard-metric-grid log-summary-grid" aria-label="Log totals for the selected filters">
-        <span>Search</span>
+    <div><span>Matching entries</span><strong data-log-total><%= summary.total %></strong></div>
-        <input
+    <div><span>Errors</span><strong data-log-count="error"><%= summary.levels.error %></strong></div>
-          class="table-search"
+    <div><span>Warnings</span><strong data-log-count="warn"><%= summary.levels.warn %></strong></div>
-          type="search"
+    <div><span>Information</span><strong data-log-count="info"><%= summary.levels.info %></strong></div>
-          placeholder="Search logs"
+    <div><span>Debug</span><strong data-log-count="debug"><%= summary.levels.debug %></strong></div>
-          aria-label="Search logs"
+  </div>
-          data-log-search
+
-        />
+  <form method="get" action="/admin/logs" class="log-controls" data-log-filter-form>
-      </label>
+    <label>
-      <label>
+      <span>Search all matching logs</span>
+      <input
+        class="table-search"
+        type="search"
+        name="q"
+        value="<%= filters.search || '' %>"
+        placeholder="Message, details, event, or request ID"
+        aria-label="Search logs"
+        data-log-search
+      />
+    </label>
+    <label>
       <span>Severity</span>
-      <select class="table-search" data-log-level aria-label="Filter log severity">
+      <select class="table-search" name="level" data-log-level aria-label="Filter log severity">
         <option value="all" <%= filters.level === 'all' ? 'selected' : '' %>>All severities</option>
         <option value="error" <%= filters.level === 'error' ? 'selected' : '' %>>Error</option>
         <option value="warn" <%= filters.level === 'warn' ? 'selected' : '' %>>Warning</option>
-        <option value="info" <%= filters.level === 'info' ? 'selected' : '' %>>Info</option>
+        <option value="info" <%= filters.level === 'info' ? 'selected' : '' %>>Information</option>
         <option value="debug" <%= filters.level === 'debug' ? 'selected' : '' %>>Debug</option>
       </select>
-      </label>
+    </label>
-      <label>
+    <label>
+      <span>Component</span>
+      <select class="table-search" name="source" data-log-source aria-label="Filter log component">
+        <option value="all">All components</option>
+        <% if (filters.source !== 'all' && !facets.sources.some((item) => item.value === filters.source)) { %>
+          <option value="<%= filters.source %>" selected><%= filters.source %> (0)</option>
+        <% } %>
+        <% facets.sources.forEach((item) => { %>
+          <option value="<%= item.value %>" <%= filters.source === item.value ? 'selected' : '' %>><%= item.value %> (<%= item.count %>)</option>
+        <% }) %>
+      </select>
+    </label>
+    <label>
+      <span>Activity type</span>
+      <select class="table-search" name="category" data-log-category aria-label="Filter log activity type">
+        <option value="all">All activity types</option>
+        <% if (filters.category !== 'all' && !facets.categories.some((item) => item.value === filters.category)) { %>
+          <option value="<%= filters.category %>" selected><%= filters.category %> (0)</option>
+        <% } %>
+        <% facets.categories.forEach((item) => { %>
+          <option value="<%= item.value %>" <%= filters.category === item.value ? 'selected' : '' %>><%= item.value %> (<%= item.count %>)</option>
+        <% }) %>
+      </select>
+    </label>
+    <label>
       <span>Range</span>
-      <select class="table-search" data-log-range aria-label="Filter by time range">
+      <select class="table-search" name="range" data-log-range aria-label="Filter by time range">
         <option value="all" <%= filters.range === 'all' ? 'selected' : '' %>>All time</option>
         <option value="<%= 5 * 60 * 1000 %>" <%= filters.range === `${5 * 60 * 1000}` ? 'selected' : '' %>>Last 5 minutes</option>
         <option value="<%= 60 * 60 * 1000 %>" <%= filters.range === `${60 * 60 * 1000}` ? 'selected' : '' %>>Last hour</option>
@@ -37,67 +77,101 @@
         <option value="<%= 7 * 24 * 60 * 60 * 1000 %>" <%= filters.range === `${7 * 24 * 60 * 60 * 1000}` ? 'selected' : '' %>>Last week</option>
         <option value="<%= 30 * 24 * 60 * 60 * 1000 %>" <%= filters.range === `${30 * 24 * 60 * 60 * 1000}` ? 'selected' : '' %>>Last month</option>
       </select>
-      </label>
+    </label>
-      <label>
+    <label>
-      <span>Entries</span>
+      <span>Entries shown</span>
-      <select class="table-search" data-log-limit aria-label="Limit log entries">
+      <select class="table-search" name="limit" data-log-limit aria-label="Limit log entries">
-        <option value="50" <%= filters.limit === '50' ? 'selected' : '' %>>50 most recent</option>
+        <% [50, 100, 250, 500].forEach((amount) => { %>
-        <option value="100" <%= filters.limit === '100' ? 'selected' : '' %>>100 most recent</option>
+          <option value="<%= amount %>" <%= filters.limit === `${amount}` ? 'selected' : '' %>><%= amount %> most recent</option>
-        <option value="250" <%= filters.limit === '250' ? 'selected' : '' %>>250 most recent</option>
+        <% }) %>
-        <option value="500" <%= filters.limit === '500' ? 'selected' : '' %>>500 most recent</option>
       </select>
-      </label>
+    </label>
+    <div class="filter-actions button-group">
+      <button type="submit" class="button">Apply filters</button>
       <a class="button subtle" href="/admin/logs">Reset</a>
-      <a class="button subtle" href="<%= `/admin/logs?range=${encodeURIComponent(filters.range)}&level=${encodeURIComponent(filters.level)}&limit=${encodeURIComponent(filters.limit)}` %>">Refresh</a>
+      <button type="button" class="button subtle" data-log-download>Download</button>
-      <button type="button" class="button subtle" data-log-download>Download logs</button>
     </div>
+  </form>
+
+  <div class="log-live-status" data-log-live-status role="status" aria-live="polite">
+    <span class="status-indicator">Connecting to live updates</span>
+    <button type="button" class="button subtle" data-log-live-refresh hidden>Show new entries</button>
   </div>
-  <div class="log-window" data-log-list>
+
+  <div class="log-window" data-log-list data-log-limit-value="<%= filters.limit %>">
     <% if (!logs || !logs.length) { %>
-    <p class="hint">No log events yet.</p>
+      <p class="hint" data-log-empty>No logs match these filters.</p>
     <% } else { %>
-      <% logs.forEach((log) => { %>
+      <% logs.forEach((entry) => { %>
       <details
-        class="log-entry level-<%= log.level %>"
+        class="log-entry level-<%= entry.level %>"
         data-log-entry
-        data-level="<%= log.level %>"
+        data-level="<%= entry.level %>"
-        data-timestamp="<%= log.created_at %>"
+        data-source="<%= entry.source || 'core' %>"
-        data-search="<%= `${log.message} ${log.details || ""}`.toLowerCase() %>"
+        data-category="<%= entry.category || 'general' %>"
+        data-timestamp="<%= entry.created_at %>"
+        data-search="<%= `${entry.message} ${entry.details || ''} ${entry.source || ''} ${entry.category || ''} ${entry.event || ''} ${entry.request_id || ''}`.toLowerCase() %>"
       >
         <summary>
           <span class="log-marker" aria-hidden="true"></span>
-          <span class="log-message"><%= log.message %></span>
+          <span class="log-message"><%= entry.message %></span>
-          <span class="log-level-pill"><%= log.level %></span>
+          <span class="log-scope-pill"><%= entry.source || 'core' %></span>
-          <span class="log-time"><%= new Date(log.created_at).toLocaleString() %></span>
+          <span class="log-level-pill"><%= entry.level %></span>
+          <span class="log-time"><%= new Date(entry.created_at).toLocaleString() %></span>
         </summary>
-        <% if (log.details) { %>
+        <div class="log-entry-meta">
-        <pre class="log-details"><%= log.details %></pre>
+          <span><strong>Activity:</strong> <%= entry.category || 'general' %></span>
+          <% if (entry.event) { %><span><strong>Event:</strong> <%= entry.event %></span><% } %>
+          <% if (entry.request_id) { %><span><strong>Request ID:</strong> <code><%= entry.request_id %></code></span><% } %>
+        </div>
+        <% if (entry.details) { %>
+          <pre class="log-details"><%= entry.details %></pre>
         <% } else { %>
-        <div class="log-details empty">No additional details.</div>
+          <div class="log-details empty">No additional details.</div>
         <% } %>
       </details>
       <% }) %>
     <% } %>
   </div>
+
+  <details class="lumi-expandable-settings log-storage-settings">
+    <summary><span><strong>Log storage</strong><span class="hint">Retention limits and automatic cleanup</span></span></summary>
+    <div class="lumi-expandable-body">
+      <p class="hint">Lumi removes entries exceeding either limit during startup and whenever these settings are saved. Plugin data and other application records are unaffected.</p>
+      <form method="post" action="/admin/logs/retention" class="form-grid compact-grid">
+        <div class="field">
+          <label for="log-retention-days">Keep logs for up to</label>
+          <input id="log-retention-days" type="number" name="max_age_days" min="1" max="3650" value="<%= retention.maxAgeDays %>" required />
+        </div>
+        <div class="field">
+          <label for="log-retention-entries">Maximum stored entries</label>
+          <input id="log-retention-entries" type="number" name="max_entries" min="1000" max="1000000" step="1000" value="<%= retention.maxEntries %>" required />
+        </div>
+        <div class="field form-actions"><button type="submit" class="button">Save storage settings</button></div>
+      </form>
+    </div>
+  </details>
 </section>
+
 <div class="modal-backdrop" data-log-modal aria-hidden="true">
   <div class="modal">
     <div class="modal-header">
       <h3>Download logs</h3>
       <button type="button" class="icon-button" data-modal-close aria-label="Close">
-        <svg viewBox="0 0 24 24" aria-hidden="true">
+        <svg viewBox="0 0 24 24" aria-hidden="true"><path d="M6 6l12 12M18 6l-12 12" stroke="currentColor" stroke-width="2" stroke-linecap="round" /></svg>
-          <path d="M6 6l12 12M18 6l-12 12" stroke="currentColor" stroke-width="2" stroke-linecap="round" />
-        </svg>
       </button>
     </div>
     <form method="get" action="/admin/logs/download" class="form-grid">
+      <input type="hidden" name="source" value="<%= filters.source %>" />
+      <input type="hidden" name="category" value="<%= filters.category %>" />
+      <input type="hidden" name="q" value="<%= filters.search || '' %>" />
       <div class="field">
         <label>Timespan</label>
         <select name="range">
           <option value="all">All time</option>
           <option value="<%= 5 * 60 * 1000 %>">Last 5 minutes</option>
           <option value="<%= 60 * 60 * 1000 %>">Last hour</option>
-          <option value="<%= 24 * 60 * 60 * 1000 %>">Last 24 hours</option>
+          <option value="<%= 24 * 60 * 60 * 1000 %>" selected>Last 24 hours</option>
           <option value="<%= 7 * 24 * 60 * 60 * 1000 %>">Last week</option>
           <option value="<%= 30 * 24 * 60 * 60 * 1000 %>">Last month</option>
         </select>
@@ -105,22 +179,19 @@
       <div class="field full">
         <label>Severities</label>
         <div class="checkbox-grid">
-          <label><input type="checkbox" name="level" value="error" /> Error</label>
+          <% ['error', 'warn', 'info', 'debug'].forEach((level) => { %>
-          <label><input type="checkbox" name="level" value="warn" /> Warning</label>
+            <label><input type="checkbox" name="level" value="<%= level %>" /> <%= level === 'warn' ? 'Warning' : level.charAt(0).toUpperCase() + level.slice(1) %></label>
-          <label><input type="checkbox" name="level" value="info" /> Info</label>
+          <% }) %>
-          <label><input type="checkbox" name="level" value="debug" /> Debug</label>
         </div>
-        <p class="hint">Leave unchecked for all severities.</p>
+        <p class="hint">Leave unchecked for all severities. Current component, activity, and search filters are retained.</p>
       </div>
       <div class="field">
         <label>Entries</label>
-        <select name="limit">
+        <select name="limit"><option value="50">50 most recent</option><option value="100">100 most recent</option><option value="250">250 most recent</option><option value="500">500 most recent</option><option value="all">All matching entries</option></select>
-          <option value="50">50 most recent</option>
+      </div>
-          <option value="100">100 most recent</option>
+      <div class="field">
-          <option value="250">250 most recent</option>
+        <label>File format</label>
-          <option value="500">500 most recent</option>
+        <select name="format"><option value="txt">Readable text</option><option value="jsonl">Structured JSON lines</option></select>
-          <option value="all">All entries</option>
-        </select>
       </div>
       <div class="modal-actions">
         <button type="button" class="button subtle" data-modal-close>Cancel</button>

update-manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "Lumi Core",
-  "version": "0.2.10",
+  "version": "0.2.11",
   "channel": "stable",
   "released_at": "2026-07-18",
   "compatible_from": "0.1.9",
@@ -8,7 +8,7 @@
   "replaces_versions": [
     "1.2.0"
   ],
-  "migration_notes": "Includes the 1.2.0 version correction, production plugin-update diagnostics, secured read-only production diagnostics, Windows/network-share-safe plugin replacement, clobber-safe shared form submission, and Node.js 24-compatible Windows dependency startup. Lumi synchronizes runtime dependencies on restart and preserves settings, databases, plugin data, community knowledge, AI models, runtimes, uploads, logs, feedback, and secrets.",
+  "migration_notes": "Includes the 1.2.0 version correction, production diagnostics, Windows/network-share-safe updates, Node.js 24-compatible dependency startup, and structured redacted logging with automatic retention. Lumi preserves existing settings, databases, logs, plugin data, community knowledge, AI models, runtimes, uploads, feedback, and secrets.",
   "rollback_safe": true,
   "requirements": [
     "Node.js 18 or newer"
@@ -145,6 +145,18 @@
       ],
       "rollback_safe": true,
       "migration_notes": "Fixes local Windows startup and automatic dependency repair under Node.js 24; all existing data remains preserved."
+    },
+    {
+      "version": "0.2.11",
+      "channel": "stable",
+      "released_at": "2026-07-18",
+      "compatible_from": "0.1.9",
+      "migration_kind": "patch",
+      "replaces_versions": [
+        "1.2.0"
+      ],
+      "rollback_safe": true,
+      "migration_notes": "Adds structured logging columns, redaction, retention, and the enhanced administrator log viewer; existing log rows and all other local data remain preserved."
     }
   ]
 }