Skip to content

[Rust] Verify the size prefix before skipping it - #9205

Open
Nexory wants to merge 1 commit into
google:masterfrom
Nexory:rust-verify-size-prefix
Open

[Rust] Verify the size prefix before skipping it#9205
Nexory wants to merge 1 commit into
google:masterfrom
Nexory:rust-verify-size-prefix

Conversation

@Nexory

@Nexory Nexory commented Aug 23, 2026

Copy link
Copy Markdown

size_prefixed_root verifies the root that the size prefix points at, but never
reads the prefix itself. SkipSizePrefix::run_verifier advances four bytes and
hands over:

impl<T: Verifiable> Verifiable for SkipSizePrefix<T> {
    #[inline]
    fn run_verifier(v: &mut Verifier, pos: usize) -> Result<()> {
        T::run_verifier(v, pos.saturating_add(crate::SIZE_SIZEPREFIX))
    }
}

So a buffer whose prefix promises more bytes than the buffer holds verifies, and
the caller gets an accessor for it. The three other verifiers in this repository
read the prefix: C++ in VerifySizePrefixedBuffer
(include/flatbuffers/verifier.h:256), C# in
Verifier.VerifyBuffer (net/FlatBuffers/FlatBufferVerify.cs:805), and Swift
since #9196, where the review asked for it in these words: "it would be better if the
swift implementation also follows a similar convention. Where we only skip after
we have verified the prefixed value too", linking verifier.h#L256. This patch
does the same for Rust.

The doc comment on size_prefixed_root already says verification "may not be
maximally performant or catch every error (though that is the goal)". This is
one of those errors, and the parenthesis names catching them as the aim.

What it looks like

One schema, table Doc { text:string; } with root_type Doc, built with
FinishSizePrefixed and holding "hello": 36 bytes, correct prefix 32. Only the
first four bytes differ between the files below, the remaining 32 are identical:

prefix C++ VerifySizePrefixedDocBuffer Rust today Rust with this patch
32, correct accepted accepted accepted
0 accepted accepted accepted
1 accepted accepted accepted
1000000 rejected accepted, Some("hello") rejected
4294967295 rejected accepted, Some("hello") rejected

The third column is measured before the patch and the fourth after it, on the
same five files. Rust then agrees with C++ on all five.

The fix

The check mirrors the C++ one, which is an upper bound rather than an equality.
That matters, so I measured what stays accepted. Building one buffer and running
it through size_prefixed_root before and after the patch:

buffer before after
one buffer, 148 bytes accepted accepted
two concatenated, 296 bytes accepted accepted
one buffer plus 64 trailing bytes accepted accepted
prefix len - 4, correct accepted accepted
prefix len, four too many accepted rejected

A prefix smaller than the slice keeps working, which is what a caller passing a
stream or a larger read buffer relies on. C# requires exact equality here and
would reject those; this patch follows C++.

Two details a reviewer will probably ask about, both measured on the same 148
byte buffer by binary searching the smallest max_apparent_size it still
verifies under:

version smallest accepted max_apparent_size
today 251
this patch 255
using v.range_in_buffer(start, size) instead 399

The bounds check is written out rather than delegated to range_in_buffer
because that call books the whole promised size against apparent_size, which
is the 399 above: the payload would be counted twice. What remains is the four
bytes get_uoffset books for the prefix itself, which nothing read before, so
the counter sits four higher. At the default of 1 << 31 that only matters for
a buffer whose apparent size comes within four bytes of the limit, but it is a
change, not nothing.

reflection shares the entry point through size_prefixed_root_as_schema, which
forwards to size_prefixed_root. flatc --size-prefixed writes schemas that way
(FinishSizePrefixed in idl_parser.cpp) and reads them back through
VerifySizePrefixedSchemaBuffer, so the C++ side of that round trip checks the
prefix today and the Rust side does not. On a 244 byte .bfbs written by
flatc --size-prefixed, the unmodified file is accepted before and after this
patch, while the same file with the prefix set to 1000000 goes from accepted to
rejected.

Test

The test sits in tests/rust_usage_test, next to the existing
test_size_prefixed_buffer, because that is what RustTest.sh runs; the
library's own unit tests are not part of it. Without the fix it reports

size prefix 148 promises more bytes than the buffer holds, yet it verified

while test_size_prefixed_buffer keeps passing, so the two together show the
check is narrow.

bash RustTest.sh passes: serde tests, no_std compilation, rust_usage_test in
both configurations, both heap allocation checks, and the clippy step: 634
passing test results across 29 binaries, which is 317 tests run twice because
the suite covers both configurations. Nothing fails. Two steps in that script
are conditional and did not run here: the outdir tests need a built flatc
next to the checkout, and the miri run needs RUST_NIGHTLY=1. Running clippy
directly on rust/flatbuffers before and after the patch gives the same set of
messages, with one pre-existing warning shifted by eleven lines.

size_prefixed_root verifies the root that the size prefix points at, but
never reads the prefix itself: SkipSizePrefix::run_verifier advances
four bytes and hands over. So a buffer whose prefix promises more bytes
than the buffer holds verifies, and the caller gets an accessor for it.

The three other verifiers in this repository read the prefix. C++ does
so in VerifySizePrefixedBuffer, C# in Verifier.VerifyBuffer (the class
in net/FlatBuffers/FlatBufferVerify.cs), and Swift since google#9196, where
the review asked for the same convention and linked verifier.h L256.

The check mirrors the C++ one, which is an upper bound rather than an
equality, so a prefix smaller than the slice keeps working: a stream of
size prefixed buffers passed as one slice, or a larger read buffer,
still verifies. One behaviour does shift. get_uoffset counts the four
prefix bytes towards apparent_size, which nothing read before, so the
counter sits four higher: the smallest max_apparent_size that accepts a
given 148 byte buffer moves from 251 to 255. The default is 1 << 31.
Delegating to range_in_buffer instead would move it to 399, because that
books the whole promised size on top of the payload the verifier walks
anyway.

The bounds check is written out rather than delegated to
range_in_buffer, because that would add the whole promised size to
apparent_size and fail a large but legitimate prefix with
ApparentSizeTooLarge.

reflection shares the entry point through size_prefixed_root_as_schema.
flatc --size-prefixed writes schemas that way and reads them back
through VerifySizePrefixedSchemaBuffer, so the C++ side of that round
trip checks the prefix today and the Rust side does not.

The test sits in tests/rust_usage_test because that is what RustTest.sh
runs; the crate's own unit tests are not part of it.
@Nexory
Nexory requested a review from dbaileychess as a code owner August 23, 2026 18:12
@github-actions github-actions Bot added the rust label Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant