Skip to content

Honest speed and time left while a backup runs, and in Recent runs - #5

Merged
razbenaharon merged 2 commits into
mainfrom
fix-run-progress
Nov 18, 2026
Merged

razbenaharon merged 2 commits into
mainfrom
fix-run-progress

Conversation

@razbenaharon

Copy link
Copy Markdown
Owner

Why

A real backup on 2026-11-18 finished fine, but the numbers on screen were wrong:

  • Speed while running was guessed from how fast the backup drive's free space shrank. That jumps when robocopy sets a file's size before writing it, sits at zero while unchanged files are compared or deleted files are moved aside, and means nothing for versions (restic writes only compressed new data).
  • Time left came only from restic's own guess, rounded up to whole minutes. On that run restic said 33–61 s while 25–11 s were left, so 20 s read as "About 1 min left" or "2 min".
  • Speed in Recent runs divided bytes by the whole run: 7 MB/s for a plain copy that copied at about 140 MB/s, and 397 MB/s for versions, which restic mostly passes by size and date without reading.

What changed

Status line of a running backup (DevDriveBackup.ps1 Update-RunStatus, new helpers in DevDriveBackup.Data.psm1)

  • Plain copy: files and bytes copied are read from robocopy's own log records (only the new part, every 2 s); speed is over the last 10 s. Checked against the real log: 5,150 files / 1,387,789,076 bytes, exactly robocopy's totals.
  • Versions: "Counting files" until restic has counted them all (its share done jumps from 29% to 11% before that), then "N of M files checked".
  • Time left: the usual length of the step (middle of its last three runs that went through, same source, kind and purpose), blended with the tool's own guess by how far it is, plus the usual length of the steps still waiting. Under 50 s reads "Less than a minute left"; past the usual length with nothing to go by, "Taking longer than usual". Replaying the real run: within about 2 s, where restic was off by up to 48 s.

Speed in Recent runs (Invoke-FastMirror.ps1, Get-CenterRunSpeed, Complete-CenterStep)

  • Invoke-FastMirror times each robocopy run (Seconds in summary.Runs); robocopy's own Times add up every thread under /MT.
  • The history records CopySeconds: the time of the runs that copied something.
  • Speed = CopiedBytes / CopySeconds. Versions, checks, repairs and entries from before this change show no speed. The Help page says what the number means.

For the reviewer

  • Older history entries now show an empty Speed cell: they have no copy time, and the old number was misleading.
  • Get-CenterFreeBytes stays (Get-CenterDrive uses it); it is no longer used for speed.
  • Two [math]::Max(0, fraction) calls had to take [double]0: with an int first argument PowerShell picks the integer overload and rounds the fraction to 0.
  • Codex was consulted before and after each part; its accepted findings (run folder of the previous step, check/repair history kinds, unreadable bytes) are fixed and tested.

Tests

  • Test-BackupData.ps1: 306 pass (new checks for log counting, partial lines, run-folder choice, rate window, usual length, time left, copy time and speed)
  • Test-BackupWindow.ps1: 96 pass
  • Test-BackupMirror.ps1: 171 pass
  • Test-BackupStyle.ps1: pass
  • The live status line was exercised with a simulated copy and versions step; not yet seen in a real backup.

🤖 Generated with Claude Code

razbenaharon and others added 2 commits November 18, 2026 10:10
…not from free space

The status line guessed the copy speed from how fast the backup drive's free
space shrank. That jumps when robocopy sets a file's size before writing it,
stays at zero while unchanged files are compared, and says nothing true for
versions, which write only compressed new data. The time left came only from
restic's own guess, rounded up to whole minutes: on a real run it said one or
two minutes when 10 to 25 seconds were left.

- Plain copy: files and bytes copied are read from robocopy's own log records,
  only the new part every 2 seconds; the speed is over the last 10 seconds.
- Versions: "Counting files" until restic has counted them all, then
  "N of M files checked".
- Time left: the usual length of the step (the middle of its last three runs
  that went through), blended with the tool's own guess by how far it is, plus
  the usual length of the steps still waiting. "Taking longer than usual" when
  there is nothing left to go by.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…versions show none

The Speed column divided what was copied by the length of the whole run. Most
of a plain copy is comparing files that did not change and moving deleted ones
aside, so a copy made at about 140 MB/s showed 7 MB/s. Versions divided what
restic went through by the run's length, but restic passes most files by size
and date without reading them, so 397 MB/s was no speed either.

- Invoke-FastMirror times each robocopy run (Seconds in summary.Runs);
  robocopy's own Times add up every thread under /MT.
- Complete-CenterStep records CopySeconds: the time of the runs that copied
  something, unknown when one of them has no time or no readable bytes.
- Get-CenterRunSpeed: a plain copy's speed is CopiedBytes / CopySeconds;
  versions, checks, repairs and entries from before this show none.
- The Help page says what the speed means.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@razbenaharon
razbenaharon merged commit 0ec1f93 into main Nov 18, 2026
3 checks passed
@razbenaharon
razbenaharon deleted the fix-run-progress branch November 18, 2026 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant