Repository navigation
A save put back is the other device's; emptying is no version until meant - #36
Draft
ChakraFusion wants to merge 43 commits into
Draft
ChakraFusion wants to merge 43 commits into
ChakraFusion wants to merge 43 commits into
Conversation
…te a large part of a save "Keep mine" (and "Keep both", which ends the same way) recorded every local file as shared with the peer before the peer had pulled any of it. When that pull did not finish - a large save, a request that timed out, a device that went away - the next sync read the files the peer was missing as deletions made over there, and deleted them on the device whose version had just been kept. On a save of ~240k files this removed tens of thousands; the folders then differed again, the conflict came back, and the same answer did it again. - markResolvedLocal records as shared only what both sides verifiably hold (the intersection of the two manifests). A file only this device has is new to the peer, not deleted by it. If the peer cannot be asked, nothing is recorded as shared. Same for extra save locations (ResolveRootConflict). - A sync that would delete more than 100 files and more than 10% of a save, on either side, is held as a conflict instead of applied. A deletion confirmed on purpose (the emptied-folder flow) is not held. - The conflict carries uncapped counts of what differs (diffFiles stops at 100), and the dialog says what each answer does: "Keep mine"/"Keep both" delete nothing; "Keep theirs" deletes N files only this device has, shown in red when that is many. Older builds without the counts fall back to counting the capped list. TestKeepLocal_PeerThatNeverPulled_DeletesNothingHere fails on v2.4.1 (300 files -> 100); either change alone makes it pass. Also adds tests that an interrupted pull or a peer holding part of a save never turns into deletions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…its own Performance - Batched pulls (protocol revision 2): small files are fetched up to 256 per request (4MB), 4 requests in flight, over a new /api/p2p/files route. Each file is still written through the PatchWriter (verified, fsynced, renamed). One request per file cost ~200ms on a real LAN; on loopback 5000 files went from 42s to 12s. Peers below revision 2 and relay connections keep the per-file path. - Unchanged games: when this device still holds the agreed base, the manifest request carries ifHash and a peer that still holds it too answers "unchanged" instead of sending the manifest. Older peers ignore the parameter; their full answer is used, not fetched twice. - Manifests are gzipped for clients that accept it (every Go client does, so older versions read them too): ~5.6x smaller with real hashes. - Manifests and batches get a 10-minute client timeout instead of 30s; a large manifest over a VPN never arrived inside 30s, so those games never synced. - An in-sync pass also mirrors the peer's latest snapshot, which only a pull did before; games identical everywhere from the start showed history on one device only. - A batch is bounded by what the responder counts (each requested block at the file's full block size, at most the file's real size), so a batch of many small files with a few larger ones is never refused as too large; a batch refused anyway is fetched again as two halves. Robustness - Presence: paired LAN peers are probed every 10s on their own ticker. Probes used to ride along with the minute-long reconcile, which waits for a whole sync-all pass; peers reached over a VPN (no UDP broadcast) showed online/offline minutes late. Probe rounds are serialized. - Games whose save folder is not on this device are skipped by sync-all and dropped from the retry queue instead of failing every 20s. - "syncing X with Y" is logged only when there is something to do; most passes produced only "already in sync" lines, which rotated everything else out of the log within hours. - The interrupted-pull safety test also runs over batched pulls. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every peer started each run with whatever status the database held, and a peer counts as offline only after three failed probes. For a PC switched off overnight that meant being shown online after every start until three probes - the first ten seconds after the background loops started - had failed. - The presence loop probes at once on start, then every 10s. - The three-miss tolerance is for devices heard from during this run (answered a probe, or seen by discovery, a request or the relay since start). A peer with no sign of life since start goes offline on its first unanswered probe. Adds an opt-in e2e measurement (OPENSAVE_PRESENCE_TIMING) of how long a stopped device is shown online: 30s, by design, for one seen this run.
…tches Network-aware order: - Each device records how fast its connection to every other one is: from the pulls it makes, and from a short speed test (/api/p2p/speedtest, 2 MB, at most every 6 hours) when nothing has measured it. Before anything is measured, the address kind stands in: home network fast, VPN (Tailscale's range) slower, the internet relay slowest. - A sync goes to the fastest devices first. A device that is behind takes the newer save from the fastest device holding it. - Having asked a close device to take a newer save, a sync waits for it to finish (it reports sync-complete) before asking a much slower one, bounded by how long the transfer should take. The save reaches the close devices at full speed, and the slow ones can then take it from whichever device is nearest to them. - The device list shows each connection's kind and measured speed. Block fetches on direct addresses had a fixed 30-second limit. "Direct" includes VPN addresses, and a few MB of blocks at a VPN relay's hundred-odd KB a second outlasted it: the request failed, and with it the whole pull, over and over. The limit now grows with what is asked for, as it already did over the internet relay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A speed test that failed - a peer not yet on a build that answers one - counted as done for six hours, so every link stayed unmeasured and the order fell back to address guesses all that time. A link never measured is now tested again ten minutes after a failed test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… end A pull of a large save over a slow link takes an hour, and the link's speed was recorded once, when it ended: all that time the device list said "not measured" and the sync order fell back to address guesses. The link is now measured over each 30-second window while the pull runs, and the rest when it ends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Over a home network the 2 MB speed test is over in a fraction of a second, and the rule that ignores transfers under half a second - meant for syncs, whose small transfers say more about round trips than about the link - threw it away: the fastest link stayed unmeasured, and with nothing measured it sorted behind the measured slow ones. A test that quick is now run again at 8 MB, and a test counts however short it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a save of ~240k small files (Project Zomboid map chunks) a repeat manifest build took 13-19s in 2.4.0; now ~1.3s, with an identical hash. - Walk with filepath.WalkDir instead of filepath.Walk. Walk's extra Lstat per entry opens every file on Windows and dominated a warm build. - Reuse block read buffers (sync.Pool) instead of allocating 64KB+ per file: a cold build allocated ~15GB of garbage, now ~0.6GB. - Sort with sort.Slice when evicting from the hash cache. The insertion sort was quadratic in the entry count (map order is random). - Raise the cache budget floor to 256MB and scale it at 1MB per game, ceiling 512MB. 64MB was filled by one large save on its own, so the cache evicted and re-read that save on every pass. A budget, not an allocation: small libraries never come near it. - Share the whole-file hash string with the block hash for single-block files (most save files). - FileEntryForInfo: a cache lookup with the FileInfo a walk already has.
…compressing them again Every snapshot stays a complete zip that restores on its own. What changes is how it is written: a file whose content (SHA-256, as already recorded per snapshot and held by the hash cache) matches the previous snapshot of the same branch is copied in as its already-compressed entry (zip.Writer.Copy). Only new and changed files are compressed. On a 204k-file save: a full snapshot 80.5s, a follow-up 8.9s, same size. The archive walk also moves to filepath.WalkDir for the same reason as the manifest walk.
…no warning - A missing save folder is looked for at the identical path on every other drive letter (a Steam library on D: on one PC and E: on another). Exactly one match is adopted for this device and logged; none or several are logged once. Searched at most once an hour. - A save folder on a drive this device does not have is shown as "Not on this device" (muted) instead of a warning, kept out of the warning headline and the notification bell. A folder gone from a drive that exists is still a warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
OpenSave runs in the background, and with Go's default (let the heap double before collecting) a large save's manifests showed up as twice their size in Task Manager. GOGC 50 and a 1 GiB soft memory limit make the runtime collect and return memory sooner, for a little CPU. GOGC and GOMEMLIMIT set in the environment still win. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New package internal/owntouch records the paths OpenSave itself writes, removes, creates or re-dates: pulled files, a peer's deletions, the folders a pull creates, the files "keep theirs" removes and the mtimes "keep mine" touches. The watcher classifies every event by it: a burst made only of OpenSave's own changes moves the recorded hash and nothing else - no auto-snapshot, and no OnChanged sending the sync straight back out. Before, every change a sync applied looked exactly like the game saving. A peer deleting files one request at a time produced an auto-snapshot every couple of seconds, each of a save that was neither the old one nor the new one; they filled the retention budget and pushed out the snapshots that mattered. - a file counts as OpenSave's only if nothing wrote it after the mark (its mtime is not later), so a game saving straight after a sync is still the game's change. - once OpenSave is done with a file (renamed into place, re-dated to the peer's time) it records the time and size it left it with, and from then on the file is OpenSave's only while it has exactly those - a game saving within the two seconds the time comparison allows is still the game's. - a path found gone counts as OpenSave's only if OpenSave removed it (MarkRemoved): a file it pulled and the game or a person then deleted is their change. - the rescan after new folders come under watch is OpenSave's own when it created those folders (a pull of a save with subfolders), and someone else's otherwise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The batched pull re-dates each file to the peer's time like the per-file pull does, and like it marks the file first and settles it after, so the watcher takes a batch for OpenSave's own change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every device now records, per game, which version of the save it holds as a version vector: one counter per device, moved only when the save really changes there (never by a sync; see internal/owntouch). The version travels in the manifest response, and two devices compare versions before anything else: - one older than the other: that device is behind. It takes the newer save whole - every file and every deletion - and only from a device holding it completely; only then does it adopt that version. Until then it is a source for nobody: nothing is taken from it, pushed by it, or deleted on its word. A pull that stops part-way is remembered across restarts, so the half-copy is never mistaken for a change made there. - a device remembers the newest version it has heard of (its target). Two devices that both know a newer one exists do nothing with each other until a device holding it is back. This is what stopped a half-finished copy being passed on, with everything it lacked read as deletions. - an empty folder holds no version, so it fills itself and never raises a conflict. - neither newer: both changed independently. Only this asks. Resolution stays mutual: "keep mine" on one device does not make the other's save simply older - the other device is asked in turn, and the first is not asked again (the answer travels with the version). If both keep their own, the later answer stands. Every sync and every manifest served first checks the save against the hash recorded with its version, so a change the watcher has not reported yet (it waits for the game to finish writing) is a version before anyone compares - otherwise a device with unrecorded work would look merely behind and take the other's save over it. Changes to excluded files are not versions, and neither is writing an exclusion rule: the recorded hash is tagged with the rules it was taken under and simply re-taken when they change. A device fetching back a save it was asked to put back after it was emptied (hold.go) follows the hold's rules. Saves from before versions get one named after their content, so devices that already agreed agree at once; ones that did not are compared as before until they agree. Games with automatic sync off are not watched, so they keep the old file comparison; so do peers that predate this. Also: - the mass-deletion guard is removed. It held syncs that would delete many files for a decision - a stopgap for deletions read from a half-finished copy, which versions now rule out at the source. - the sync engine marks what it writes and removes (owntouch), and a burst of deletions a peer asks for takes one safety snapshot, at most every ten minutes, instead of one auto-snapshot per file.
…is save everywhere"
Saves that existed before version tracking get a starting version named
after their content, and nothing recorded says which device's is newer.
Between two of those, the old file-by-file comparison used to decide - and
it either asked, or merged the two file by file into a save no game wrote.
Now, between saves from before versions:
- if the two last agreed on a state and one still holds exactly it, the
other is newer;
- otherwise if one is newer in every file that differs (newer, or missing on
the other side) and the other holds no file written after its newest,
that one is newer;
- the older device takes the newer save whole, never file by file. The side
that gives way was older in every differing file, so it loses nothing
newer than what it takes.
- anything else is asked about.
"Use this save everywhere" (game header, POST /api/games/{id}/use-everywhere)
is for states nobody can rank - half-arrived copies, mixtures. The device's
version then covers every save from before versions (entry "g:*"), so every
such device takes it whole, keeping a snapshot first. A change made on
another device since versions existed is not overruled: that is a conflict.
An open conflict no longer blocks a game for good once a version newer than
both sides of it arrives: it was settled elsewhere, and is cleared.
…amage
The transition to save versions happens one device at a time, and a device
still on an older build decides by the old file comparison: whatever its copy
lacks it reads as deleted, so a copy that arrived half-way spread its gaps as
deletions to the devices that had the whole save. That is how a restored save
lost 6,927 files again to a device not yet updated.
- Nothing is taken from a peer that does not keep save versions (status
peer_outdated_app); it can still take from this device.
- Deletions it asks for are refused, on the LAN and over the relay. Requests
from builds that keep versions say so ("versioned": true).
- The device list marks such a peer "needs update".
- The conflict dialog offers "Use this device's save everywhere" for when
neither side is the right save.
Peers signal that they keep versions with "versions": true in the manifest
answer (and LAN protocol revision 3). The relay path now also carries the
save's version: until now, relay syncs never got one and always fell back
to the old file comparison.
… not asked about Before versions, devices that never ran a game held copies relayed from the others at different times, and syncs file by file had left some holding mixtures. Two such copies, neither newer in every file, were asked about - a question about states nobody made, whose right answer is always the copy with the newer work. Now, after the two existing rules (unchanged since the last agreed state; newer in every differing file), the save holding the most recent work wins: the latest-written of the files where the two differ, then the one newer in more of them. The other takes it whole, never file by file, keeping a snapshot of its own first. Only an exact tie is asked about. Changes made since versions existed are not decided this way: they are versions of their own, and a real conflict between them is still asked.
A save kept as a single file is keyed by its name, so the same save named differently on two devices (an emulator's <profile>.sav) shares no path with the other. The rule settling saves from before versions by their most recent work then made the newer one the save, and the other device would have removed its own file for one with a different name. Saves with no file in common are not two versions of the same files: they are asked about. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"<peer> has an older version - asking it to take this one" was logged on every pass while the peer was still taking it, every 20 seconds during a long pull. Logged once per state now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nished Taking a peer's version checks, once its files are written, that the save is now exactly the peer's; if not, the pull counts as stopped part-way and is resumed on the next sync. A game saving in that moment - after the pull wrote its file, before the check - made the save differ, and the resumed pull then replaced the new save with the peer's. Only the snapshot taken before replacing it still held it, and both devices called themselves in sync. (CI: TestSwitch_TheSameGameUnderDifferentIDsSyncs, about one run in fifteen locally.) When every difference is a file the pull wrote or removed and someone other than OpenSave has written it since (owntouch), the pull did finish: the device takes the peer's version, records the change as a version of its own made from it, and asks the peer to take that. Anything else is still a pull stopped part-way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hange here The check after a pull counted only files the pull had written as possibly changed since. A file the pull did not touch - the same on both sides when the pull was planned - deleted here the moment the pull was done made the save differ from the peer's, the pull counted as stopped part-way, and the resumed pull fetched the deleted file back (CI: TestFullSyncFlow, Windows). A file the pull did not touch that differs now was changed here meanwhile: it counts like a file the pull wrote that someone else has written since. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Taking a newer version after following a peer onto a branch read the save back through the gate the same sync still held, and waited for itself until its time ran out (30 minutes). It reads it directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The API lists a branch's snapshots oldest first, so the newest - what a branch switch puts back - is the last, not the first its comment meant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pull of a save of many files overflowed the watcher's event queue again and again, and each overflow was taken for the game saving: an automatic snapshot of a half-pulled save every few seconds - 76 on one device for one pull - which pushed the complete ones out of retention. - The watcher asks whether a sync is writing the game (BeingWritten: now, or in the last ten seconds). If so, an overflow is taken for the sync's, and the burst is looked at only once the sync has finished. - A snapshot that could not read every file is discarded, not kept: it would stand in the history as a save that never existed. The watcher leaves the change unrecorded and takes it again at its next pass; a sync that needed the copy before replacing files does not replace them. - OpenSave's own temporary files (*.opensave.tmp) are never archived. - A sync that finds both sides already identical records the peer's latest snapshot only for a game with no history here at all. It recorded one per peer: a full copy of the same save for every device, gigabytes for a save of a quarter-million files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snapshot is now named by what it holds: a hash over every file's name in the archive and its content hash - the hashes the sync compares, so the same save gives the same value on every device (snapshot.ContentKey, recorded in snapshots.content_hash). - A snapshot taken for a sync with a peer is not archived when this device already has a snapshot of exactly those files: the peer sends what its snapshot holds (SnapshotInfo.ContentHash), and its id is recorded as an alias of the one here (snapshot_aliases), so it is not asked about again. Without that from an older peer, the archive is made, named, and dropped if it duplicates one. - A copy taken before a sync replaces the save is the snapshot that already holds those files, when nothing automatic would remove that one first (pinned, taken by hand, or the branch's newest). - Snapshots that already exist are named and merged (at start, two minutes in, and POST /api/snapshots/merge-duplicates): of copies of the same files on a branch, a pinned one is kept, else one taken by hand, else the newest; pinned ones are never removed; the others' ids become aliases. - A peer's snapshot that could not read every file is discarded, like any other incomplete snapshot. One save of 780 MB had been archived six times on one device. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An automatic snapshot every file of which a newer snapshot on the same branch holds unchanged is only a step on the way there - a pull that stopped part-way, a save that only gained files - and restoring it gives nothing the newer one lacks but the absence of what was added since. The snapshot flood of an interrupted pull left dozens of these per game. They are removed after identical copies are merged (PruneContained), at start and on POST /api/snapshots/merge-duplicates. The other way round is kept: a newer snapshot contained in an older one is a save something was deleted from, on purpose. Pinned snapshots and ones taken by hand are never removed. A snapshot without a recorded file list has it read from its archive once, and recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… they arise - Older automatic snapshots that a newer one on the same branch holds entirely - a pull that stopped part-way, a save that only gained files - are removed: the largest snapshots are tried first as the ones holding others, passes repeat until one removes nothing, and from then on each new snapshot (its own or a peer's) is checked against the older ones as it is taken (PruneOnCreate, set by the daemon). A full pass runs only for games with snapshots whose content is not named yet; POST /api/snapshots/merge-duplicates runs it for all. - Never the other way round: a newer snapshot contained in an older one is a save something was deleted from on purpose. Pinned and hand-taken snapshots are never removed. - A snapshot removed for one holding exactly the same files leaves its id as an alias: restoring, listing files, previewing and comparing by it reach the snapshot kept. Deleting does not follow aliases. - A copy taken before replacing the save stands in for an existing snapshot of the same files only when that one is the branch's newest - what a branch switch puts back. Fixes found on the way: - The watcher waits out only a sync writing the game now, not the seconds after: a change the game made straight after a pull was snapshotted late. - Taking a newer version after following a peer onto a branch read the save back through the gate the same sync still held, and waited for itself until its time ran out (30 minutes). It reads it directly. - e2e: branchHolds read the oldest snapshot (the API lists them oldest first), not the newest its comment says; tests that counted snapshots of unchanged saves now change the save first or check what is recoverable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A session ended and its "After playing" snapshot was taken; the game's last write, settling in the watcher a moment later, made an "Auto backup" of the same save, and duplicate pruning then removed the older one - the session's name with it (CI: TestSession_EndingOneKeepsTheSaveAndSyncsIt, Windows). When a newer snapshot holding exactly the same files takes an older one's place, it takes over the older one's reason if it has only the default "Auto backup", and its note if it has none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snapshot replaced while the check ran - a newer one holding the same files took its place - was reported as damaged: its archive was gone. It is skipped instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found in a four-device retest: - Steam rewrites remotecache.vdf in a game's userdata folder on every start. It counted as a new save: an auto-snapshot of a game nobody played, a version bump, and every other device trying to take it - failing every 30 s where the folder is under Program Files. It is the launcher's file, not the save: delta.NeverSynced leaves it out of manifests and snapshots, and the watcher does not wake for it. - The snapshot recorded after taking a peer's version zipped the whole save again. It goes through the snapshot manager now, which reuses the unchanged files of the previous snapshot (ArchiveSaveTo); a large save no longer sits at "100%" for minutes after the download. - A save folder that cannot be written (access denied) is said once, plainly, with what to do, and pulls into it wait 30 minutes instead of failing on every pass from every device. - An identical older snapshot is aliased before it is deleted, and Restore looks an id up again if its row vanished in between, so a restore racing the prune never finds nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0672b53 left Steam's remotecache.vdf out of the manifest itself. On a device whose tracked folder held nothing else - every game tracked at Steam's userdata/<appid> folder - the first start of that build saw the folder emptied: "every save file was deleted", an empty auto-snapshot, and a new version of its own for every such game, which every other device would have done at the same moment. - The served manifest lists the file again: a device on an older build must not read the gap as a deletion and remove its own copy. It is left out where decisions are made instead, exactly like an exclusion rule: filterManifest, the lineage, recorded deletions and the version hash all leave it out, with or without rules of the game's own. - The version hash tag includes the list (delta.NeverSyncedList), so a new list re-takes the hash instead of naming a version. - Once per change of the list, at start, each game's recorded hashes are re-taken in the new terms when the save is the one they were recorded for (watcher.ContentHashBeforeNeverSynced, AdoptNeverSyncedView), so the watcher does not take the upgrade for a change. Recorded in a new marks table. - A hold recorded over such a file lets go: it is not something that has to come back. - The snapshot check skips a snapshot removed while it ran (a newer one with the same files took its place) instead of calling it damaged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A game that keeps its settings beside its saves had one PC's graphics settings handed to every other: Crimson Desert's save/user_engine_option_save.xml, among others. The Ludusavi manifest already says which files are a game's settings: it tags them "config", as opposed to "save" - 16,700 entries. The index now keeps those (indexedGame.Config, index format v3), and for each tracked game the ones that lie inside its save folder become exclusion patterns relative to it (presets.DeviceSettings), matched component by component so a template's <storeUserId> wildcard meets the folder actually tracked. An entry tagged both is a save. A pattern that would cover the whole folder is never applied: a wrong exclusion loses saves, a missed one only syncs a setting. They are applied as the game's exclusion rules are, through one function everything that applies rules now takes them from (syncengine.Engine.IgnoreText): never synced in either direction, never deleted on another device, still in every snapshot, and kept on this device by a restore. The game's own rules come after them, so a "!" line brings one back; a new per-game switch (games.sync_device_settings, "Sync these settings between devices too" in the configuration tab, with the files listed) syncs them all. The patterns change without anyone writing a rule - a manifest update, a first start with this build - so at start each game's recorded hashes (last snapshot, version, base agreed with each device) are carried over to the new view where the save is the one they were recorded for (daemon.adoptExclusionView, generalising the never-synced step). Toggling the switch goes through the same path as editing the rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The game database names the settings files of the games it knows. For the
rest, a file is now taken for a game's settings by its contents
(presets/detectsettings.go): the vocabulary of one machine's display -
resolution, vsync, shadow quality, upscaler - matched inside identifiers
split at case changes and separators ("_enableFrameGeneration",
"r_materialAniso"), in UTF-8 or UTF-16. Unreal Engine's Config/<platform>
folder is taken by convention.
Names only add weight, by how reliably the Ludusavi manifest's own labels
show them to mean settings: measured over its 16,700 settings and 9,000
save entries, "settings" names 1,521 settings files and 70 saves, ".ini"
1,831 and 137, so a name never decides on its own. A name with a save
word in it rules a file out unless the contents are overwhelming, the
game database's save locations are never taken, and nor are logs,
binaries, long files that are only partly settings, or Syncthing's
conflict copies.
A file taken for settings that changes like a save is given back for
good (verdict "save", which also overrules the game database):
- before it is left out, and at every later run, the game's snapshots
are asked whether it changed in most of the snapshots in which the rest
of the save changed (snapshot/history.go);
- while it is left out, every automatic snapshot here and every manifest
another device sends count whether it changed when the rest of the save
did; three such changes, outnumbering the times it stayed put, give it
back (daemon/detectsettings.go). Counted continuously, since retention
and duplicate merging leave most games with too few snapshots to judge.
Detected files and verdicts are kept per game (settings_files,
settings_observations) and applied through the device settings, so the
hashes recorded under the old list are carried over as before. The game's
file list says why each excluded file is excluded (its own rule, the game
database, or detected), and unticking one there syncs it.
On this machine: 74 files in 25 games detected, among them Crimson
Desert's, DOOM Eternal's and Elden Ring's settings, and 12 games the game
database does not cover; no file the snapshot history shows changing with
the save.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
relativeUnder split paths only on the system's own separator, so on Linux a pattern written Windows-style (Ludusavi's data, a Proton prefix) never matched the folder it names, and path.Match read its backslashes as escapes. Both separators are split on, on every system. Found by CI on Linux. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A game autosaves every few minutes, and each autosave was synced as it came: on one afternoon a device took eleven versions of Crimson Desert in 26 minutes from the device it was being played on, each with its own "synced from another device" snapshot, and the playing device snapshotted every autosave too. The gameplay guard only waits while save files are locked, and many games do not keep them locked. And a device took other devices' versions of a game running on it, writing a save under the game. While a game is being played on a device (the daemon's play sessions): - it is not synced either way there: SyncGame declines it quietly (ErrPlayingHere, which counts as ErrHeld for callers that only want to know it was not an error), so nothing is taken for a running game; - another device asking for it is answered "playing" (PlayingMessage), both for its manifest and for deletions, and logs that once instead of failing; the answer contains SettlingMessage, so a device on an older build takes it as "busy, ask again"; - the watcher snapshots a change to it at most once per 30 minutes (HoldSnapshot), so a crash still leaves a recent copy without one snapshot per autosave. When the session ends, the existing "After playing" snapshot is taken and the save goes to the other devices in one sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… play
A 20-minute session of Crimson Desert was never seen as one: its process
keeps its path from everyone (protected against tampering), so the
session tracker, which matches running programs by path, saw nothing,
and every autosave was snapshotted and synced as it came.
- A process whose path cannot be read is listed by its file name, and
matched against the programs found in the game's install folders
(Target.Programs, three folders deep, installers, crash reporters,
redistributables and anti-cheat services left out).
- A save that keeps changing while no session is seen is handled as play
(daemon/sessions.go, noteActivity): one change on its own is kept and
synced at once, as before - a game that saves as it closes; a second
within 10 minutes begins a stretch that is not synced either way, with a
checkpoint at most every 30 minutes, and once the save has stayed
unchanged for 10 minutes it is kept ("After playing") and synced, as a
session's end does. Said once in the log, per game, when it happens.
Options.SyncEveryChange turns the stretch off for the e2e harness, whose
tests of syncing change a save several times a minute and expect every
change to travel.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A session of Crimson Desert installed at G:\Games\Crimson Desert was never seen: session detection knew install folders only from Steam's libraries and the folders games are usually kept in, and the game's entry read "not found on this PC". Its program path was readable all along. - A running program inside a folder named as one of the game's install folders is that game, wherever the folder is: the names the game database knows (Ludusavi's installDir, "Crimson Desert"), and the game's own name when it is two words or a long one - "Rust" is a folder in every Rust toolchain. Generic folder names never count. - The folder a session was seen running from is remembered (marks), so the game is known to be installed there, and its programs can be told by name from then on. - A change made while no session is seen is held from the first, not the second: it is kept and synced once the save has stayed unchanged for 5 minutes, longer than the gap between Crimson Desert's first two autosaves (3.6 minutes), so an unseen session sends one version, at its end. A second change within 10 minutes makes it a stretch that ends after 10 minutes unchanged, as before. - A change waiting out the watcher's debounce already counts as play: a sync landing in those two seconds sent one change on just before it was held (watcher.ChangePending). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From a session of Crimson Desert on a device that now recognised it:
- A process named CrimsonDesert.exe stayed behind for 15 minutes after the
game closed, with under 1 MB of memory and no processor time, and held
the session - and with it the save - open. A process is now measured
(memory, processor time; sessions.Proc), and one holding under 32 MB
that used no processor time since the last poll is not the game
(sessions.Idle, daemon.activeProcesses). One that cannot be measured
still counts.
- A game whose sessions this device has seen (a session recorded, or the
folder it ran from known) is not held back while it is closed: a change
to its save then is not play, and is kept and synced at once; so is a
change within two minutes of a session's end. The 5 and 10 minute holds
stay for games never seen running.
- A snapshot taken while a game is played is named "Checkpoint while
playing", not left as an unnamed automatic one ("Save changed"), and the
session's checkpoints are removed once its end is kept: one snapshot per
session. Pinned ones stay; after a crash they are what is left.
- In a session, a change no longer also starts a stretch: the stretch's
timer ended inside the session, cleared its last checkpoint, and from
half an hour in let nearly every autosave through as a checkpoint -
"Save changed" every ten minutes in a session that was seen.
- A stretch waits for twice the longest gap between the game's changes
when that is longer than its usual wait (at most 45 minutes), and the
gap is remembered per game: a game that autosaves every ten minutes is
not let through at every autosave.
- The e2e harness prints a daemon's log when a test fails.
- "not found on this PC" under a game's title is a link that asks for the
game's program, which then says where it is installed and what runs
when it is played.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crimson Desert, killed, left two processes behind that kept their memory and used no processor time at all, and its session stayed open: idle was only what held under 32 MB as well. Each process's processor time is followed from poll to poll; one that has used none for a minute is not the game being played, however much memory it holds. A game being played uses some all the time - a paused one still draws its menu. One that cannot be measured still counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestRunning_ByInstallFolderName gave processes Windows paths; a process's program is a native path, so the test now builds them for the system it runs on. Found by CI on Linux. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…are no saves
Twenty-odd games were tracked at Steam's userdata/<user>/<appid> folders
that held nothing but Steam's own remotecache.vdf - no save, nothing to
sync. Measuring a found save now leaves out the files no save is made of
(delta.NeverSynced), so such a folder is empty and is never offered or
tracked as a new game.
Tracking by hand gets the game database's knowledge, which a folder picked
on its own lacked - name only, no App ID, so no cover art, no launching,
no session detection, no device settings:
- "Track game" beside "Track folder" on Home: a game running now (a
program holding over 200 MB that the database knows), one found by
name, or the one a program or a save folder chosen belongs to. It is
tracked with its name, App ID, program, and a save location the database
knows on this device (presets/gamedb.go, /api/gamedb/{running,search,
identify}).
- "Track folder" recognises the folder the same way and tracks it with the
game's name and App ID ("Recognised as ...").
- A program is recognised by the Steam install folder it is in, or a folder
above it named as the game's install folder, compared without spaces:
the database writes "ProjectZomboid" for "Project Zomboid". Session
detection compares install folder names the same way.
- A game re-tracked on another device is not restored at a folder here
that holds no save (only Steam's own file): placed afresh, it follows
the other device's save location instead of coming back at the empty
folder it was untracked for.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tracking Escape From Duckov from the game database put a second game on ...\TeamSoda\Duckov\Saves, inside the folder an entry named "Duckov" already tracked: the same files watched, snapshotted and synced twice. Only the very same folder was refused. - A new game is not tracked at a folder inside a tracked game's folder, or holding one; the refusal names the game (TrackGame, overlappingGame). Moving a game's own folder is not affected. - "Track game" says when the game picked is tracked already under another entry - its folder is, holds or is held by one the game database knows, or it has the App ID - and offers to link that entry to it instead: the database's name, App ID and program, without tracking the folder twice. - A game tracked without an App ID is linked to the game database once, when its folder is the game's save location, lies inside it, or holds it one folder up (IdentifyFolderStrictly) - never a folder holding many games' saves. It gets cover art, launching, session detection and its device settings; its name stays as it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eant Putting Project Zomboid back after its folder was emptied (test B on four devices) ended with the restored save as a version of its own on the device it was put back on, which every other device was asked to take, and with a file from the local snapshot winning over the other device's newer copy because its date was later. And it waited ten minutes first. - An emptied save is not a version while nobody has said the emptying was meant (emptyingUnconfirmed): neither the watcher's change nor a refresh names it one, and nothing written while it is held back does. Only "delete them there too" makes it a version the other devices take. Put back, the device keeps the version it had, so no device is told it is behind. - While a save is put back, every file both devices hold differently is the other device's, whatever the dates say; nothing is offered from here, nothing deleted there, and no conflict is asked about (putBackDecision). - An emptied save, or one held back after being emptied, is never handled as play: no stretch holds the answer back, and OpenSave's own restore is not taken for a game writing its save. - The window says what is happening: putting back from which snapshot, fetching the rest (with progress) or waiting for another device to be online; and when changes are held back while a game seems to be played. - A device asked for a game whose folder is not on it now (a drive not plugged in) says so, as a device with no folder for it yet does, instead of an error the asking device logged every few seconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@ChakraFusion is attempting to deploy a commit to the sivadaboi's projects Team on Vercel. A member of the Team first needs to authorize it. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #21.
Stacked on the track-game PR (#35). 1 new commit, about 180 changed lines.
Why
Putting Project Zomboid back after its folder was emptied, tested on four PCs:
What
emptyingUnconfirmed). Neither the watcher nor a refresh names it one, and nothing written while it's held back does. Only "delete them there too" makes it a version for the other devices. When the save is put back, the device keeps the version it had, so no device is told it's behind.putBackDecision).Tests
🤖 Generated with Claude Code