Skip to content

Fix HTTP file mismatch (CRC:00000000) when a resource file is re-downloaded - #5150

Open
QueryOfficial wants to merge 1 commit into
multitheftauto:masterfrom
QueryOfficial:fix/checksum-of-file-being-written
Open

Fix HTTP file mismatch (CRC:00000000) when a resource file is re-downloaded#5150
QueryOfficial wants to merge 1 commit into
multitheftauto:masterfrom
QueryOfficial:fix/checksum-of-file-being-written

Conversation

@QueryOfficial

Copy link
Copy Markdown
Contributor

Summary

Client-side file reads could return an empty buffer while reporting success, which made a
freshly downloaded resource file checksum to CRC:00000000 MD5:D41D8CD98F00B204E9800998ECF8427E
— the checksum of zero bytes. Three changes:

  1. CDownloadableResource::GenerateClientChecksum() now retries briefly (4 × 20 ms) when a
    file reads as empty and the server-reported download size is non-zero. A file that does not
    exist yet checksums to CChecksum() instead, so files still awaiting download never enter this
    path and pay no cost.
  2. SharedUtil::FileLoadWithTimeout() and SharedUtil::FileLoad(std::nothrow, …) took the
    number of bytes to read from GetFileAttributesExW and returned true with an empty buffer when
    that reported 0. Sizing a read from the directory entry samples the length and the bytes at two
    different moments. Both now open the handle first and size the read with GetFileSizeEx, and a
    short read is never reported as success. FileLoadWithTimeout moves the whole open/measure/read
    sequence onto the single worker thread that already provided the hang timeout — this keeps the
    protection from 7bc2315 / 7e69069 end-to-end and drops thread spawns per call from 3 to 2.
    FileLoad(std::nothrow, …) is what CClientDFF/CClientTXD/CClientColModel use, so the same
    defect could load a model from an empty buffer even when the checksum passed.
    The POSIX branch switches from stat64(path) to fstat64(fd) for the same reason.
  3. CChecksum's Windows client cache is now validated and populated with size/mtime taken from
    the open handle, never caches a zero-byte result, and gains
    InvalidateChecksumCacheEntry(path) — called wherever a file is deleted or replaced
    (CPacketHandler::Packet_ResourceStart, CResource::VerifyPendingClientChecksums,
    CResourceFileDownloadManager::DownloadFinished, CSingularFileDownload).

CResource::Load() also re-reads once before reporting a mismatch, so a lost race becomes a silent
retry rather than a player-visible error.

GetFileAttributesExWithTimeout() had no callers left after this and was removed, along with the
WIN32_FILE_ATTRIBUTE_DATA forward declaration that existed only for it.

Motivation

Resolves #5120.

Restarting a resource that contains .dff / .txd / .hf / .mp3 files produced
HTTP server file mismatch! with Got CRC:00000000 on every restart, and the model was not
replaced. A second restart of the same resource worked.

That MD5 is the MD5 of an empty input, so the client was hashing zero bytes for a file that had
downloaded correctly. Client scripts were unaffected because CResource::Load() reads those through
FileLoad() (stdio, length from the open stream); only the auto-download files went through
GenerateChecksumFromFileUnsafe()FileLoadWithTimeout().

Test plan

New unit testsTests/client/SharedUtilFile_Tests.cpp, 11 cases covering both loaders: whole
file, genuinely empty file, missing file, offset/maxSize, repeated delete + rewrite, a writer handle
still open, and GetFileInfoWithTimeout. Full suite: 315/315 pass
(Bin/tests/Tests_Client_d.exe).

Measured before/after with a standalone harness running the pre-fix and post-fix
implementations side by side against a writer thread doing delete → create → write → close, counting
calls that returned success with a buffer that was not the complete payload:

Scenario before after after + retry
A: static complete file (3000 calls) 0 0
D: writer handle open, unflushed (3000 calls) 0 0
B: concurrent rewrite (20000 calls) 1736 567 0
C: concurrent rewrite, 1 ms zero-byte window (3000 calls) 2208 1751 0

A and D show the read is fine once the file is settled; B and C are the failure mode, and the retry
closes it completely.

Checklist

  • Your code should follow the coding guidelines.
  • Smaller pull requests are easier to review. If your pull request is beefy, your pull request should be reviewable commit-by-commit.

@FileEX FileEX added the bugfix Solution to a bug of any kind label Aug 4, 2026
@FileEX FileEX added this to the 1.7 (Current) milestone Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Solution to a bug of any kind

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CRC mismatch after every resource restart since the last update

2 participants