Skip to content

feat(assets): add conservative explicit cleanup for local files - #854

Open
Zhao0335 wants to merge 16 commits into
pascalorg:mainfrom
Zhao0335:fix/asset-storage-orphan-cleanup
Open

Zhao0335 wants to merge 16 commits into
pascalorg:mainfrom
Zhao0335:fix/asset-storage-orphan-cleanup

Conversation

@Zhao0335

@Zhao0335 Zhao0335 commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Scope

Adds conservative, explicit cleanup for local asset:// Files related to #733. This PR does not automatically delete a File when a node is removed, and it does not fully fix #733. IndexedDB is shared by every scene and tab on the origin; deleting from one active scene could corrupt another saved scene that references the same URL.

Changes

  • @pascal-app/core provides deleteAsset, listLocalAssetUrls, sweepLocalAssetsExcept, and collectors for nested asset:// references in nodes and material maps. deleteAsset also revokes its cached object URL.
  • apps/editor/lib/local-asset-gc.ts provides an explicit host API. It collects references from the current graph, the localStorage draft, and every server scene. A failed, truncated, or malformed scene inventory (or unparseable local draft) aborts the sweep. The live graph is read again immediately before deletion; both nodes and materials are included, including unsaved texture references added during the inventory walk.
  • Automatic mount-time and per-node deletion paths were removed after review. The explicit API has no automatic caller. A host must invoke it only when it can ensure a complete scene inventory, no in-flight writes or uploads, and enough API rate budget.

Review questions

  • Delete then recreate / shared URL: No timer runs after a node delete, so recreating a node or sharing a URL cannot race a delayed delete. Explicit GC keeps any URL found in the full persisted inventory or the live graph. There is no per-node or cross-scene reference counter yet.
  • Page reload: There is no pending timer to survive reload. Orphans remain until a host explicitly runs GC under the conditions above.

Validation

  • bun test apps/editor/lib/local-asset-gc.test.ts — 8 passed, including an IndexedDB File referenced by a live material added during inventory and malformed-inventory guard cases.
  • bun run --cwd apps/editor check-types — passed.
  • bunx biome check apps/editor/lib/local-asset-gc.ts apps/editor/lib/local-asset-gc.test.ts — passed.
  • bunx turbo run build --filter='./apps/editor^...' — passed (existing Turbopack dynamic filesystem tracing warning).

Note

Medium Risk
Adds IndexedDB deletion APIs, but sweeps are gated on a complete scene inventory and the GC entrypoint has no automatic caller; misuse could still remove files referenced by scenes outside the built keep-set.

Overview
Introduces opt-in garbage collection for origin-wide IndexedDB asset:// files instead of deleting on node remove/replace, because the same URL can be shared across scenes (#733).

@pascal-app/core adds deleteAsset, listLocalAssetUrls, and sweepLocalAssetsExcept, plus deep collectors (collectNodeAssetUrls, collectSceneAssetUrls, collectGraphAssetUrlsFromParts) that walk nodes and material texture maps for asset:// strings. Deletes also revoke cached blob object URLs.

apps/editor/lib/local-asset-gc.ts exposes runLocalAssetGc, which unions references from the live graph, the localStorage draft, and every server scene (sequential GETs). It returns null and skips the sweep when the scene list fails, is malformed, hits the 500-scene list cap, or localStorage JSON is invalid—so partial keep-sets never delete files. The live graph is read again immediately before sweeping so mid-run uploads and unsaved material textures stay protected.

Editor UI paths (reference/site panels) no longer imply immediate local file cleanup on guide replace/delete; comments document that physical deletion must go through the explicit GC (or future core hooks). runLocalAssetGc is not wired to mount or autosave. Tests use fake-indexeddb for sweep and safety cases.

Reviewed by Cursor Bugbot for commit 88f1dce. Bugbot is set up for automated code reviews on this repo. Configure here.

Deleting a scan or guide node only routed http(s) URLs to onDeleteAsset.
Local asset:// Files (up to 200 MB per scan) stayed in IndexedDB forever.

- Add deleteAsset + sweepOrphanAssets to @pascal-app/core asset storage
- Schedule delayed local deletes on node removal (undo-safe grace period)
- Sweep unreferenced assets after applySceneGraphToEditor
- Cover storage, lifecycle, undo-restore, and cancel paths with tests

Fixes pascalorg#733
@pascal

pascal Bot commented Sep 11, 2026

Copy link
Copy Markdown

I hit an error while handling your request (Model unavailable on AI Gateway free tier: Free tier users do not have access to this model. Upgrade to paid credits at https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fai%3Fmodal%3Dtop-up for unrestricted…).

Please try again, rephrase, or reach out if it keeps failing.

Error id: 92c31b27-f9c4-4ce1-a2fb-ce2e00b06d8f

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/editor/src/lib/scene.ts Outdated
IndexedDB is origin-global and not scoped by project/scene. Sweeping every
asset_data: entry missing from the just-loaded graph deleted local Files
still referenced by other scenes, blank canvases, and previews.

Keep only the undo-safe delayed delete when a node is removed. Fixes the
orphaned-File leak from pascalorg#733 without a multi-scene wipe.

Addresses review on pascalorg#854.
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Addressed the Bugbot finding in d44b09d: removed the scene-load orphan sweep. IndexedDB is origin-global, so sweeping entries missing from the just-loaded graph could wipe assets still referenced by other scenes. Cleanup is now only the undo-safe delayed delete of a specific �sset:// File after its last live node is removed.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/editor/src/lib/local-asset-lifecycle.ts Outdated
The grace-period re-check only inspected the active useScene graph.
Switching projects during the window could delete an origin-global
IndexedDB File that another saved scene still referenced.

Bump a scene epoch in applySceneGraphToEditor; timers scheduled under a
previous epoch refuse to delete.

Addresses review on pascalorg#854.
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Addressed the latest Bugbot finding: delayed deletes now capture a scene epoch at schedule time. �pplySceneGraphToEditor bumps the epoch; timers from a previous epoch refuse to deleteAsset. Switching projects during the grace window no longer deletes origin-global IndexedDB Files that another saved scene still references. Covered by a new unit test.

@Zhao0335

Copy link
Copy Markdown
Contributor Author

This finding is already fixed on the current HEAD (cb97731), which is ahead of the reviewed 7ae66a2.

  • �pplySceneGraphToEditor bumps a scene epoch (�umpLocalAssetSceneEpoch).
  • scheduleLocalAssetDelete captures that epoch at schedule time; the timer returns without deleteAsset if the epoch changed (project/scene switch).
  • Covered by unit test: does not delete after a scene switch (another graph may still use the File).

No further code change needed for this comment.

@Aymericr Aymericr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker: cleanup is attached to two UI handlers rather than the deletion lifecycle, so the bug still reproduces through supported delete paths.

For example, packages/editor/src/hooks/use-keyboard.ts deletes a selected guide/scan directly with useScene.getState().deleteNode(...), and the generic selection/delete surfaces in selection-routing.ts and group-actions.ts also bypass scheduleLocalAssetDelete. Deleting the same local reference with the Delete key therefore still leaves its asset:// File in IndexedDB.

Please move the scheduling to a single path that observes all committed node removals/replacements, or route every supported guide/scan deletion path through one shared helper. Add a regression test for keyboard deletion (and ideally delete-mode/generic selection deletion) so this cannot remain UI-entry-point dependent. The 120-second grace period, live-reference recheck, cache revocation, and scene-epoch guard are otherwise sensible.

Cleanup was attached to the Site panel and reference-panel handlers, so
keyboard Delete (use-keyboard.ts), selection delete, MCP, and group
actions still orphaned asset:// Files in IndexedDB (pascalorg#733).

- Move grace-period scheduling into @pascal-app/core
- Hook deleteNodesAction and applyNodeChanges delete/update paths
- Cover keyboard-style deleteNode with a core regression test

Fixes the remaining delete paths called out in review.
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Addressed the deletion-lifecycle blocker in $(git rev-parse --short HEAD):

  • Local �sset:// scheduling now lives in @pascal-app/core and runs from deleteNodesAction (and �pplyNodeChanges deletes/url replacements), not the Site/reference panel handlers.
  • Covers keyboard Delete (use-keyboard.ts → deleteNode), selection delete, MCP delete_node, and group actions.
  • Regression test: packages/core/src/lib/local-asset-lifecycle-delete.test.ts — keyboard-style deleteNode schedules the File; a second live reference keeps it.

Grace period, live-reference recheck, cache revocation, and scene-epoch guard are unchanged.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/core/src/store/actions/node-actions.ts Outdated
Comment thread packages/editor/src/components/ui/panels/reference-panel.tsx Outdated
Comment thread packages/editor/src/lib/scene.ts Outdated

@Aymericr Aymericr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The central delete lifecycle now covers the missing keyboard/generic paths, but the current head still has three correctness blockers before cleanup is safe: (1) URL replacement schedules deletion without a fire-time scan of all live nodes, so undo or another node sharing the same asset:// URL can lose its File; (2) the reference-panel replacement schedules before updateNode commits, so a no-op/throw can delete the still-current asset; and (3) the scene epoch only advances in applySceneGraphToEditor, while unload/import/reset paths can replace the graph through setScene/clearScene and leave an old timer valid. Please make the timer predicate query the current graph for the URL at fire time, schedule replacements only after a committed change, and put epoch invalidation at the shared graph-replacement boundary. CI also needs the three Biome organize-import fixes shown by quality. Add regressions for shared-reference replacement, undo restoration, failed/no-op replacement, and a clear/set scene transition.

…Scene

Address review on pascalorg#733 cleanup:

- URL-replacement timers re-check the live graph at fire time (undo /
  shared asset:// handles)
- Guide replace no longer schedules delete before updateNode commits
- Epoch invalidation lives on unloadScene/setScene, not only
  applySceneGraphToEditor
- organizeImports on the three CI-flagged files

Regressions: shared-reference replace, undo restore, unreferenced
replace, setScene epoch.
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Addressed the three correctness blockers and Biome organizeImports on $head:

  1. Replacement timers re-query the live graph for the URL at fire time (collectSceneAssetUrls(get().nodes)), so undo or a duplicate guide sharing the same �sset:// handle keeps its File.
  2. Guide replace no longer schedules before updateNode; only the core updateNodes path schedules after the url change commits.
  3. Epoch is bumped in unloadScene and setScene (shared graph-replacement boundary), not only �pplySceneGraphToEditor.

Regressions added for shared-reference replacement, undo restoration, unreferenced replacement, and setScene epoch. packages/core: 1545 tests pass.

@Aymericr Aymericr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new fire-time live-graph checks fix the reported undo/shared-node cases, but there is still a cross-scene data-loss blocker. asset:// storage is origin-global, and scene/project duplication preserves those URLs. If scene A and an unopened saved scene B reference the same File, deleting the last reference in the currently loaded A schedules deleteAsset; staying in A for 120 seconds means the timer sees no current reference and deletes B’s File. The epoch only protects a scene switch during the grace window, not references in persisted scenes that were never loaded. This is the same ownership problem that made the original sweep unsafe.

Please do not physically delete origin-global Files based only on the active graph. The cleanup path needs a durable cross-scene reference index/refcount (updated atomically with scene persistence), per-scene asset ownership/copying, or a conservative explicit garbage-collection pass that can inspect every saved graph. Until one of those exists, this PR trades an orphan leak for silent corruption of another project. Add a regression with two persisted scenes sharing one asset URL; deleting from one must not break the other.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/core/src/store/actions/node-actions.ts Outdated
Move bumpLocalAssetSceneEpoch import before material-library and put
events/bus value export before its type export.
IndexedDB asset:// Files are origin-global and survive scene duplication,
so deleting based on the loaded graph alone corrupts other saved scenes
(pascalorg#733 review).

- Remove automatic deleteAsset from deleteNodes/updateNodes timers
- Core exposes sweepLocalAssetsExcept(keepUrls) for explicit GC
- apps/editor GC unions current + localStorage + every server scene;
  skips the sweep when any source cannot be enumerated
- Regression: two persisted scenes sharing one URL — delete from one
  must not break the other

Fixes the cross-scene data-loss blocker.
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Addressed the cross-scene data-loss blocker in 55e04ae.

No longer deletes origin-global Files from the active graph.

  1. Removed automatic deleteAsset from deleteNodes / updateNodes timers.
  2. Core now exposes explicit GC: sweepLocalAssetsExcept(keepUrls).
  3. Standalone editor GC (apps/editor/lib/local-asset-gc.ts) builds the keep-set from current scene + localStorage scene + every /api/scenes graph. If any source cannot be enumerated, it skips the sweep rather than partial-deleting.
  4. Regression test: two persisted scenes share one asset:// URL; sweeping with both keep-sets keeps the File.

Orphan leak may remain until a successful full enumeration runs — that is preferred over silent corruption of another project.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread apps/editor/lib/local-asset-gc.ts
Comment thread apps/editor/components/scene-loader.tsx Outdated
- Treat a full /api/scenes page (limit=500, no cursor) as incomplete —
  never sweep when older scenes may still reference a File
- Re-read useScene nodes immediately before sweepLocalAssetsExcept so
  uploads during the long per-scene fetch stay in the keep-set
- Keep initialScene urls in the extra keep-set

Addresses Bugbot findings on 55e04ae.
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Fixed the two High findings on 0023ec0:

  1. Truncated inventory — /api/scenes has no cursor; a full page (500) is treated as incomplete and GC skips instead of sweeping. Listing failure still skips.
  2. In-flight uploads —
    unLocalAssetGc takes a live-node getter and re-reads useScene immediately before sweepLocalAssetsExcept, and keeps initialScene urls in the extra keep-set.

Unit tests cover truncated list, failed list, and mid-GC live-graph growth.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread apps/editor/lib/local-asset-gc.ts
Comment thread apps/editor/lib/local-asset-gc.ts
Mount-time GC races keepalive flush, other tabs, and uploads that have
reached IndexedDB but not a node; sequential per-scene GETs also burn
the autosave rate bucket.

Keep runLocalAssetGc as an explicit host API only — never called from
SceneLoader. Prefer leftover orphans over deleting live Files.
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Addressed the mount-time GC findings on 3cc6f50:

  • Removed automatic runLocalAssetGc from SceneLoader mount.
  • runLocalAssetGc remains an explicit host API only (documented: no in-flight writes, complete inventory, spare rate budget).
  • Sequential per-scene GETs no longer run on page open, so they cannot starve autosave's rate bucket.

This leaves possible orphans in IndexedDB rather than deleting Files another tab, keepalive flush, or unpersisted upload still needs.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/core/src/lib/local-asset-lifecycle.ts
Top-level url/src missed item.asset.*, captureSession.manifestUrl, and
materials texture maps. Deep-walk nodes (depth-capped) and include
graph.materials via collectGraphAssetUrlsFromParts.
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Fixed keep-set completeness on 276e5b0:

  • collectNodeAssetUrls now deep-walks the node (depth 8, cycle-safe) so item.asset.src / thumbnail, captureSession.manifestUrl, and other nested asset:// strings are included.
  • New collectGraphAssetUrlsFromParts also walks graph.materials texture maps.
  • Unit tests cover nested item fields and materials maps.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 276e5b0. Configure here.

Comment thread apps/editor/lib/local-asset-gc.ts
@dkbbdev

dkbbdev commented Sep 17, 2026

Copy link
Copy Markdown

Reviewing this against #733 — verified the fix routes local asset:// URLs into the delayed-delete scheduler so deleting a scan/guide node no longer orphans its IndexedDB File (the ~200 MB case is the one that matters on memory-constrained devices).

Questions while reading:

  1. Delete-then-recreate race — if a user deletes a scan node and immediately re-adds a guide referencing the same underlying File before the grace period elapses, does the scheduled delete get cancelled or does it remove the File out from under the new node?
  2. Ref-counting scope — two nodes sharing one asset URL is the obvious next failure mode; is that handled or explicitly out of scope here? If out of scope, a short note in the PR body would help future readers.
  3. Minor: the scheduler timer survives page reload? (IndexedDB File pending delete + reload → does the pending entry get re-scheduled or leaked until the next delete?)

@Zhao0335 Zhao0335 changed the title fix(editor): clean up orphaned local asset:// files from IndexedDB feat(assets): add conservative explicit cleanup for local files Sep 23, 2026
@Zhao0335

Copy link
Copy Markdown
Contributor Author

Thanks for the careful questions. The PR had changed substantially since its original description, so I corrected the title and body to match the current code.

  1. Delete then recreate: There is no scheduled delete on node removal now. The earlier timer was removed after the cross-scene data-loss review, so recreating a reference during a grace period cannot race it.
  2. Shared URL: There is no refcount. The explicit GC path gathers references from the live graph, the localStorage draft, and all server scenes; if the server inventory fails or is truncated, it skips deletion. b8a6af37 also makes it retain live material textures, including those added while the inventory is running. The new regression uses a real IndexedDB File.
  3. Reload: There is no pending timer to resume. Orphans remain until a host explicitly invokes GC when it can ensure a complete inventory and no in-flight writes.

This means the PR is groundwork for #733, not a complete automatic fix. I removed the closing Fixes #733 language from the description and made that limitation explicit. Local targeted tests, TypeScript, Biome, and the dependency build pass; GitHub CI is running on the new head.

@Zhao0335

Copy link
Copy Markdown
Contributor Author

@Aymericr The review threads are resolved, and the latest push fixes the live-material keep-set and makes GC abort on malformed or incomplete scene inventories. All current checks pass. Could you take another look? The PR description now states clearly that this is an explicit cleanup API and does not fully close #733.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Local asset storage never cleans up: deleting an asset:// scan or guide orphans its File in IndexedDB

3 participants