Writable overlay for the mount #5

New issue

Open

opened 2026-10-13 13:06:26 +00:00 by tacomagick · 1 comment

tacomagick commented

2026-10-13 13:06:26 +00:00

Owner

Allow writes to the mount, kept separate from the read-only Hydrus data.

Idea: the read-only Hydrus index is the lower layer. A writable directory per mount is the upper layer. Reads check the upper layer first, then fall through to Hydrus. New files and writes go to the upper layer. Deleting a Hydrus file needs a whiteout marker so it does not reappear. Editing a Hydrus file needs a copy-up of the original bytes.

Notes:

The upper layer should be a plain directory. If a user wants a VHDX or image, they mount it themselves and point the config at it. Do not implement container handling; it needs admin rights and differs per platform.
No pack/unpack on mount or unmount. Files stay as normal files, so a crash cannot lose them.
Hydrus names are generated from tags and change on refresh. Shadow copies and whiteouts must be keyed by file hash, not by path, or they get orphaned when a file is renamed.
fuser and WinFsp both support the write calls we need. We refuse them today, so the semantics are ours to implement.

Phases:

Writable upper for new files and dirs, plus whiteouts for deletes. This is enough for Syncthing.
Hash-keyed shadowing so edits and deletes survive Hydrus renames.
Copy-up on edit, plus rename, link and xattr.

If this lands, tools like Syncthing can create their own marker files, and the virtual markers from #1 become less necessary.

Allow writes to the mount, kept separate from the read-only Hydrus data. Idea: the read-only Hydrus index is the lower layer. A writable directory per mount is the upper layer. Reads check the upper layer first, then fall through to Hydrus. New files and writes go to the upper layer. Deleting a Hydrus file needs a whiteout marker so it does not reappear. Editing a Hydrus file needs a copy-up of the original bytes. Notes: - The upper layer should be a plain directory. If a user wants a VHDX or image, they mount it themselves and point the config at it. Do not implement container handling; it needs admin rights and differs per platform. - No pack/unpack on mount or unmount. Files stay as normal files, so a crash cannot lose them. - Hydrus names are generated from tags and change on refresh. Shadow copies and whiteouts must be keyed by file hash, not by path, or they get orphaned when a file is renamed. - fuser and WinFsp both support the write calls we need. We refuse them today, so the semantics are ours to implement. Phases: 1. Writable upper for new files and dirs, plus whiteouts for deletes. This is enough for Syncthing. 2. Hash-keyed shadowing so edits and deletes survive Hydrus renames. 3. Copy-up on edit, plus rename, link and xattr. If this lands, tools like Syncthing can create their own marker files, and the virtual markers from #1 become less necessary.

tacomagick commented

2026-10-13 13:09:31 +00:00

Author

Owner

this is going to be a headache