Fix plugin updates with locked data directories

This commit is contained in:
Franz Rolfsvaag 2026-07-18 17:17:58 +02:00
parent 328d06662f
commit 9647f11bf9
14 changed files with 137 additions and 144 deletions

View File

@ -1,5 +1,10 @@
# Lumi changelog # Lumi changelog
## 0.2.6
- Fixed plugin updates on Windows and network shares when a running plugin keeps its preserved `data` directory open.
- Plugin updates and automatic restores now leave preserved data in place while transactionally replacing only plugin code, with rollback if a code-file replacement fails.
## 0.2.5 ## 0.2.5
- Fixed diagnostics behind local/private reverse proxies by accepting forwarded HTTPS only from loopback, RFC1918/link-local IPv4, or private/link-local IPv6 proxy addresses. - Fixed diagnostics behind local/private reverse proxies by accepting forwarded HTTPS only from loopback, RFC1918/link-local IPv4, or private/link-local IPv6 proxy addresses.

View File

@ -678,6 +678,8 @@ This section is for Lumi chat answer feedback and AI Improvement Center work, no
## Done ## Done
- 2026-07-18: Fixed private reverse-proxy HTTPS recognition for production diagnostics in core 0.2.5 without globally trusting client forwarding headers, and taught the local client to consume the repository's env-style `.secrets` file and short diagnostic variable names directly. - 2026-07-18: Fixed private reverse-proxy HTTPS recognition for production diagnostics in core 0.2.5 without globally trusting client forwarding headers, and taught the local client to consume the repository's env-style `.secrets` file and short diagnostic variable names directly.
- 2026-07-18: Fixed Windows/network-share plugin updates in core 0.2.6 by leaving live preserved plugin data in place and transactionally replacing only code, including automatic code rollback coverage.
- 2026-07-18: Retired the one-off core repair ZIP build scripts after repository updates proved reliable; core 0.2.6 is distributed through the normal immutable Git release flow.
- 2026-07-18: Updated production diagnostics examples in core 0.2.4 to derive the full endpoint and client base URL from the administrator's current Lumi request instead of showing a placeholder hostname. - 2026-07-18: Updated production diagnostics examples in core 0.2.4 to derive the full endpoint and client base URL from the administrator's current Lumi request instead of showing a placeholder hostname.
- 2026-07-18: Fixed the shared submit-action resolver in core 0.2.3: ordinary buttons now inherit their parent form endpoint unless they explicitly declare `formaction`, restoring timed diagnostics-key creation and preventing async update actions from posting back to the Updates page. - 2026-07-18: Fixed the shared submit-action resolver in core 0.2.3: ordinary buttons now inherit their parent form endpoint unless they explicitly declare `formaction`, restoring timed diagnostics-key creation and preventing async update actions from posting back to the Updates page.
- 2026-07-18: Added production-stage plugin update diagnostics in core 0.2.2: selected plugin source is verified before snapshotting, failures record their exact stage and target in update state, and the affected plugin row displays the server error directly. - 2026-07-18: Added production-stage plugin update diagnostics in core 0.2.2: selected plugin source is verified before snapshotting, failures record their exact stage and target in update state, and the affected plugin row displays the server error directly.

View File

@ -76,6 +76,11 @@ Before a core, plugin, repo, or ZIP update, Lumi creates a snapshot under
- major-crossing flag - major-crossing flag
- compressed storage size - compressed storage size
For plugin updates, Lumi leaves the plugin's `data` directory in its existing
location and transactionally replaces only code files. This allows running
Windows services and network shares to retain open AI model/runtime handles
without blocking the update or its automatic rollback.
New snapshots contain only the replaceable rollback payload: compressed core or New snapshots contain only the replaceable rollback payload: compressed core or
plugin code and a compressed SQLite database backup. Preserved local data is plugin code and a compressed SQLite database backup. Preserved local data is
not duplicated into snapshots. In particular, LLM model files, managed AI not duplicated into snapshots. In particular, LLM model files, managed AI

View File

@ -14,7 +14,7 @@ editable: false
Lumi is the core web UI and bot runtime. Lumi is the core web UI and bot runtime.
## Runtime ## Runtime
Package: lumi-bot Package: lumi-bot
Version: 0.2.5 Version: 0.2.6
## Routes ## Routes
- GET /api/events - GET /api/events
- POST /api/destructive-confirmations - POST /api/destructive-confirmations

4
package-lock.json generated
View File

@ -1,12 +1,12 @@
{ {
"name": "lumi-bot", "name": "lumi-bot",
"version": "0.2.5", "version": "0.2.6",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "lumi-bot", "name": "lumi-bot",
"version": "0.2.5", "version": "0.2.6",
"dependencies": { "dependencies": {
"adm-zip": "^0.5.12", "adm-zip": "^0.5.12",
"better-sqlite3": "^11.5.0", "better-sqlite3": "^11.5.0",

View File

@ -1,6 +1,6 @@
{ {
"name": "lumi-bot", "name": "lumi-bot",
"version": "0.2.5", "version": "0.2.6",
"private": true, "private": true,
"type": "commonjs", "type": "commonjs",
"scripts": { "scripts": {
@ -13,7 +13,6 @@
"verify:updates": "node scripts/verify-release-metadata.js && node scripts/verify-update-system.js", "verify:updates": "node scripts/verify-release-metadata.js && node scripts/verify-update-system.js",
"verify:diagnostics": "node scripts/verify-production-diagnostics.js", "verify:diagnostics": "node scripts/verify-production-diagnostics.js",
"diagnostics:production": "node scripts/production-diagnostics-client.js", "diagnostics:production": "node scripts/production-diagnostics-client.js",
"build:repair-patch": "node scripts/build-core-repair-patch.js && node scripts/verify-core-repair-patch.js",
"verify:web-auth": "node scripts/verify-web-auth.js", "verify:web-auth": "node scripts/verify-web-auth.js",
"verify:destructive-actions": "node scripts/verify-destructive-actions.js", "verify:destructive-actions": "node scripts/verify-destructive-actions.js",
"verify:overlays": "node scripts/verify-overlays.js", "verify:overlays": "node scripts/verify-overlays.js",

View File

@ -2,6 +2,36 @@
"schema_version": 1, "schema_version": 1,
"channel": "stable", "channel": "stable",
"releases": [ "releases": [
{
"version": "0.2.6",
"ref": "refs/tags/v0.2.6",
"released_at": "2026-07-18",
"installable": true,
"rollback_safe": true,
"replaces_versions": [
"1.2.0"
],
"data_policy": "preserve",
"dependency_policy": "sync_on_restart",
"migration_notes": "Replaces plugin code without renaming preserved plugin data, allowing updates while Windows or a network share holds runtime/model files open.",
"plugins": {
"auto-vc": "0.1.6",
"birthday": "0.1.3",
"economy-framework": "0.2.10",
"economy-games": "0.1.7",
"expression-interaction": "0.2.1",
"lumi_ai": "0.8.2",
"moderation": "0.1.5",
"okf": "0.1.1",
"quotes": "0.1.2",
"sample-plugin": "0.1.0",
"throne_wishlist": "0.1.2",
"welcome_messages": "0.1.1"
},
"tools": {
"lumi_ai_web_search": "0.1.1"
}
},
{ {
"version": "0.2.5", "version": "0.2.5",
"ref": "refs/tags/v0.2.5", "ref": "refs/tags/v0.2.5",

View File

@ -1,68 +0,0 @@
const crypto = require("crypto");
const fs = require("fs");
const path = require("path");
const AdmZip = require("adm-zip");
const root = path.join(__dirname, "..");
const destination = path.join(root, "dist", "lumi-core-1.2.0-to-0.2.5-repair.zip");
const files = [
"CHANGELOG.md",
"README.md",
"package.json",
"package-lock.json",
"release-index.json",
"run.js",
"update-manifest.json",
"docs/updates.md",
"docs/production-diagnostics.md",
"docs/update-audit-0.2.0.md",
"knowledge/core/lumi-core.md",
"scripts/verify-release-metadata.js",
"scripts/verify-all.js",
"scripts/build-core-repair-patch.js",
"scripts/verify-core-repair-patch.js",
"scripts/verify-update-system.js",
"scripts/verify-production-diagnostics.js",
"scripts/production-diagnostics-client.js",
"scripts/verify-webui.js",
"src/services/dependency-manager.js",
"src/services/overlay-connectors.js",
"src/services/repo-update.js",
"src/services/production-diagnostics.js",
"src/services/settings.js",
"src/services/update-index.js",
"src/services/update-manager.js",
"src/services/update-repository.js",
"src/web/server.js",
"src/web/public/app.js",
"src/web/views/admin-updates.ejs",
"src/web/views/admin-diagnostics.ejs"
];
const hashes = {};
const zip = new AdmZip();
for (const relativePath of files) {
const filePath = path.join(root, relativePath);
if (!fs.statSync(filePath, { throwIfNoEntry: false })?.isFile()) {
throw new Error(`Repair patch input is missing: ${relativePath}`);
}
const contents = fs.readFileSync(filePath);
hashes[relativePath] = crypto.createHash("sha256").update(contents).digest("hex");
zip.addFile(relativePath.replace(/\\/g, "/"), contents);
}
const manifest = {
schema_version: 1,
name: "Lumi core 0.2.5 repair",
target: "core",
from_versions: ["0.1.9", "1.2.0", "0.2.0", "0.2.1", "0.2.2", "0.2.3", "0.2.4", "0.2.5"],
to_version: "0.2.5",
data_policy: "preserve",
dependency_policy: "sync_on_restart",
created_at: new Date().toISOString(),
files: hashes
};
zip.addFile("patch-manifest.json", Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`));
fs.mkdirSync(path.dirname(destination), { recursive: true });
zip.writeZip(destination);
console.log(destination);

View File

@ -1,62 +0,0 @@
const assert = require("assert");
const crypto = require("crypto");
const fs = require("fs");
const os = require("os");
const path = require("path");
const AdmZip = require("adm-zip");
const { verifyPatchPackage } = require("../src/services/update-manager");
const root = path.join(__dirname, "..");
const archivePath = path.join(root, "dist", "lumi-core-1.2.0-to-0.2.5-repair.zip");
assert.equal(fs.existsSync(archivePath), true, "build the repair patch first");
const zip = new AdmZip(archivePath);
const entries = zip.getEntries().filter((entry) => !entry.isDirectory);
const names = new Set(entries.map((entry) => entry.entryName.replace(/\\/g, "/")));
assert.equal(names.has("patch-manifest.json"), true);
const manifest = JSON.parse(zip.readAsText("patch-manifest.json"));
assert.equal(manifest.target, "core");
assert.equal(manifest.to_version, "0.2.5");
assert.equal(manifest.data_policy, "preserve");
assert.deepEqual(manifest.from_versions, ["0.1.9", "1.2.0", "0.2.0", "0.2.1", "0.2.2", "0.2.3", "0.2.4", "0.2.5"]);
const forbidden = /^(?:data|plugins|node_modules|config|storage|uploads|logs|database|databases|knowledge\/(?:community|corrections))(?:\/|$)|^\.env(?:\.|$)|^\.secrets$/;
for (const entry of entries) {
assert.doesNotMatch(entry.entryName, forbidden, `repair includes preserved data path ${entry.entryName}`);
}
for (const [relativePath, expected] of Object.entries(manifest.files)) {
assert.equal(names.has(relativePath), true, `${relativePath} is listed but missing`);
const actual = crypto.createHash("sha256").update(zip.readFile(relativePath)).digest("hex");
assert.equal(actual, expected, `${relativePath} checksum`);
}
assert.equal(Object.keys(manifest.files).length + 1, entries.length, "every repair file must be checksummed");
assert.equal(JSON.parse(zip.readAsText("package.json")).version, "0.2.5");
const simulation = fs.mkdtempSync(path.join(os.tmpdir(), "lumi-repair-simulation-"));
try {
const sentinels = [
"data/app.db",
"plugins/okf/data/local.db",
"node_modules/local-only/package.json",
"knowledge/community/people.md",
"knowledge/corrections/fix.md",
".env"
];
for (const sentinel of sentinels) {
const target = path.join(simulation, sentinel);
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.writeFileSync(target, `preserve:${sentinel}`);
}
zip.extractAllTo(simulation, true);
assert.equal(verifyPatchPackage(simulation).to_version, "0.2.5");
for (const sentinel of sentinels) {
assert.equal(fs.readFileSync(path.join(simulation, sentinel), "utf8"), `preserve:${sentinel}`);
}
assert.equal(JSON.parse(fs.readFileSync(path.join(simulation, "package.json"), "utf8")).version, "0.2.5");
assert.equal(fs.existsSync(path.join(simulation, "src", "services", "dependency-manager.js")), true);
assert.equal(fs.existsSync(path.join(simulation, "src", "services", "production-diagnostics.js")), true);
assert.equal(fs.existsSync(path.join(simulation, "src", "web", "views", "admin-diagnostics.ejs")), true);
} finally {
fs.rmSync(simulation, { recursive: true, force: true });
}
console.log(`Core repair patch verification passed (${entries.length} files, no preserved data payloads).`);

View File

@ -24,7 +24,18 @@ try {
fs.ftruncateSync(descriptor, 3 * 1024 * 1024 * 1024); fs.ftruncateSync(descriptor, 3 * 1024 * 1024 * 1024);
fs.closeSync(descriptor); fs.closeSync(descriptor);
const originalRenameForLockedRoot = fs.renameSync;
fs.renameSync = (from, to) => {
if (from === target) {
throw Object.assign(new Error("simulated running plugin data lock"), { code: "EPERM" });
}
return originalRenameForLockedRoot(from, to);
};
try {
replacePluginDirectory(source, target, { preserveData: true }); replacePluginDirectory(source, target, { preserveData: true });
} finally {
fs.renameSync = originalRenameForLockedRoot;
}
assert.equal(fs.readFileSync(path.join(target, "index.js"), "utf8"), "module.exports = 'new';"); assert.equal(fs.readFileSync(path.join(target, "index.js"), "utf8"), "module.exports = 'new';");
assert.equal(fs.existsSync(path.join(target, "stale.js")), false); assert.equal(fs.existsSync(path.join(target, "stale.js")), false);
@ -37,7 +48,7 @@ try {
fs.writeFileSync(path.join(failedSource, "index.js"), "module.exports = 'must-not-land';"); fs.writeFileSync(path.join(failedSource, "index.js"), "module.exports = 'must-not-land';");
const originalRename = fs.renameSync; const originalRename = fs.renameSync;
fs.renameSync = (from, to) => { fs.renameSync = (from, to) => {
if (String(from).includes(".target.update-") && to === target) { if (String(from).includes(".target.update-") && path.basename(from) === "index.js") {
throw Object.assign(new Error("simulated install lock"), { code: "EPERM" }); throw Object.assign(new Error("simulated install lock"), { code: "EPERM" });
} }
return originalRename(from, to); return originalRename(from, to);

View File

@ -4,8 +4,8 @@ const path = require("path");
const { findSafeTarget } = require("../src/services/versioning"); const { findSafeTarget } = require("../src/services/versioning");
const root = path.join(__dirname, ".."); const root = path.join(__dirname, "..");
const releaseVersion = "0.2.5"; const releaseVersion = "0.2.6";
const previousCoreVersion = "0.2.4"; const previousCoreVersion = "0.2.5";
const earliestCompatibleCoreVersion = "0.1.9"; const earliestCompatibleCoreVersion = "0.1.9";
const changedPlugins = { const changedPlugins = {
"auto-vc": { from: "0.1.5", to: "0.1.6", knowledge: "auto-vc" }, "auto-vc": { from: "0.1.5", to: "0.1.6", knowledge: "auto-vc" },
@ -87,4 +87,4 @@ assert.equal(webSearch.minimum_lumi_version, "0.2.0");
assert.equal(webSearch.minimum_lumi_ai_version, changedPlugins.lumi_ai.to); assert.equal(webSearch.minimum_lumi_ai_version, changedPlugins.lumi_ai.to);
assert.equal(hasVersionHeading(readText("plugins/lumi_ai_web_search/CHANGELOG.md"), webSearch.version), true); assert.equal(hasVersionHeading(readText("plugins/lumi_ai_web_search/CHANGELOG.md"), webSearch.version), true);
console.log("Release metadata verification passed: core 0.2.5 and 11 changed plugin/tool packages."); console.log("Release metadata verification passed: core 0.2.6 and 11 changed plugin/tool packages.");

View File

@ -16,7 +16,7 @@ function readJson(relativePath) {
const releaseIndex = readJson("release-index.json"); const releaseIndex = readJson("release-index.json");
const releaseVersions = releaseIndex.releases.map((release) => release.version); const releaseVersions = releaseIndex.releases.map((release) => release.version);
assert.deepEqual(releaseVersions, ["0.2.5", "0.2.4", "0.2.3", "0.2.2", "0.2.1", "0.2.0", "0.1.9"]); assert.deepEqual(releaseVersions, ["0.2.6", "0.2.5", "0.2.4", "0.2.3", "0.2.2", "0.2.1", "0.2.0", "0.1.9"]);
assert.equal(new Set(releaseVersions).size, releaseVersions.length, "release versions must be unique"); assert.equal(new Set(releaseVersions).size, releaseVersions.length, "release versions must be unique");
for (const release of releaseIndex.releases) { for (const release of releaseIndex.releases) {
assert.equal(normalizeRepositoryRef(release.ref), release.ref); assert.equal(normalizeRepositoryRef(release.ref), release.ref);
@ -37,6 +37,7 @@ for (const [toolId, version] of Object.entries(currentRelease.tools)) {
const baseTarget = { const baseTarget = {
current_version: "0.2.4", current_version: "0.2.4",
available_versions: [ available_versions: [
{ version: "0.2.6", ref: "refs/tags/v0.2.6", rollback_safe: true },
{ version: "0.2.5", ref: "refs/tags/v0.2.5", rollback_safe: true }, { version: "0.2.5", ref: "refs/tags/v0.2.5", rollback_safe: true },
{ version: "0.2.4", ref: "refs/tags/v0.2.4", rollback_safe: true }, { version: "0.2.4", ref: "refs/tags/v0.2.4", rollback_safe: true },
{ version: "0.2.3", ref: "refs/tags/v0.2.3", rollback_safe: true }, { version: "0.2.3", ref: "refs/tags/v0.2.3", rollback_safe: true },
@ -65,7 +66,7 @@ const corrected = buildStatus({
channel: "stable" channel: "stable"
}); });
assert.equal(corrected.version_correction, true); assert.equal(corrected.version_correction, true);
assert.equal(corrected.safe_target_version, "0.2.5"); assert.equal(corrected.safe_target_version, "0.2.6");
assert.equal(corrected.update_available, true); assert.equal(corrected.update_available, true);
assert.equal(corrected.blocked, false); assert.equal(corrected.blocked, false);

View File

@ -693,6 +693,15 @@ function replacePluginDirectory(rootPath, targetDir, options = {}) {
try { try {
fs.mkdirSync(staging, { recursive: true }); fs.mkdirSync(staging, { recursive: true });
copyDirectory(rootPath, staging, ignore); copyDirectory(rootPath, staging, ignore);
// A running plugin may legitimately keep files below data open. In
// particular, Lumi AI starts llama-server with data/runtime as its working
// directory. Windows and SMB then reject renaming the plugin root even
// though none of the code being updated is locked. Keep the data directory
// in place and transactionally replace only the plugin's code entries.
if (options.preserveData && fs.existsSync(targetDir)) {
replacePluginCodeInPlace(staging, targetDir, backup);
return;
}
if (fs.existsSync(targetDir)) { if (fs.existsSync(targetDir)) {
moveDirectory(targetDir, backup, "backup_plugin"); moveDirectory(targetDir, backup, "backup_plugin");
targetMoved = true; targetMoved = true;
@ -730,6 +739,55 @@ function replacePluginDirectory(rootPath, targetDir, options = {}) {
} }
} }
function replacePluginCodeInPlace(staging, targetDir, backup) {
const movedExisting = [];
const installed = [];
let rollbackFailed = false;
fs.mkdirSync(backup, { recursive: true });
try {
for (const entry of fs.readdirSync(targetDir, { withFileTypes: true })) {
if (entry.name === "data") continue;
moveDirectory(
path.join(targetDir, entry.name),
path.join(backup, entry.name),
"backup_plugin_code"
);
movedExisting.push(entry.name);
}
for (const entry of fs.readdirSync(staging, { withFileTypes: true })) {
moveDirectory(
path.join(staging, entry.name),
path.join(targetDir, entry.name),
"install_plugin_code"
);
installed.push(entry.name);
}
fs.rmSync(backup, { recursive: true, force: true });
} catch (error) {
try {
for (const name of installed.reverse()) {
fs.rmSync(path.join(targetDir, name), { recursive: true, force: true });
}
for (const name of movedExisting) {
const saved = path.join(backup, name);
if (!fs.existsSync(saved)) continue;
const live = path.join(targetDir, name);
fs.rmSync(live, { recursive: true, force: true });
moveDirectory(saved, live, "restore_plugin_code");
}
} catch (restoreError) {
rollbackFailed = true;
error.message = `${error.message} Plugin code rollback also failed: ${restoreError.message}. Backup: ${backup}. Staging: ${staging}`;
}
throw error;
} finally {
if (!rollbackFailed) {
fs.rmSync(staging, { recursive: true, force: true });
fs.rmSync(backup, { recursive: true, force: true });
}
}
}
function moveDirectory(source, target, operation) { function moveDirectory(source, target, operation) {
runFileOperationWithRetries(() => fs.renameSync(source, target), { runFileOperationWithRetries(() => fs.renameSync(source, target), {
target, target,

View File

@ -1,6 +1,6 @@
{ {
"name": "Lumi Core", "name": "Lumi Core",
"version": "0.2.5", "version": "0.2.6",
"channel": "stable", "channel": "stable",
"released_at": "2026-07-18", "released_at": "2026-07-18",
"compatible_from": "0.1.9", "compatible_from": "0.1.9",
@ -8,7 +8,7 @@
"replaces_versions": [ "replaces_versions": [
"1.2.0" "1.2.0"
], ],
"migration_notes": "Includes the 1.2.0 version correction, production plugin-update diagnostics, secured read-only production diagnostics with private-proxy HTTPS support, and the shared form-action fix required by timed confirmations and repository updates. Lumi synchronizes runtime dependencies on restart and preserves settings, databases, plugin data, community knowledge, AI models, runtimes, uploads, logs, and secrets.", "migration_notes": "Includes the 1.2.0 version correction, production plugin-update diagnostics, secured read-only production diagnostics with private-proxy HTTPS support, the shared form-action fix, and Windows/network-share-safe plugin code replacement. Lumi synchronizes runtime dependencies on restart and preserves settings, databases, plugin data, community knowledge, AI models, runtimes, uploads, logs, and secrets.",
"rollback_safe": true, "rollback_safe": true,
"requirements": [ "requirements": [
"Node.js 18 or newer" "Node.js 18 or newer"
@ -85,6 +85,18 @@
], ],
"rollback_safe": true, "rollback_safe": true,
"migration_notes": "Recognizes forwarded HTTPS from local/private reverse proxies for secured diagnostics; preserved local data is not replaced." "migration_notes": "Recognizes forwarded HTTPS from local/private reverse proxies for secured diagnostics; preserved local data is not replaced."
},
{
"version": "0.2.6",
"channel": "stable",
"released_at": "2026-07-18",
"compatible_from": "0.1.9",
"migration_kind": "patch",
"replaces_versions": [
"1.2.0"
],
"rollback_safe": true,
"migration_notes": "Updates plugin code without renaming a running plugin's preserved data directory, avoiding Windows and network-share locks; preserved local data is not replaced."
} }
] ]
} }