Skip to content

use trailing/leading ones to get length of inlined strings - #14

Merged
tomtomwombat merged 2 commits into
mainfrom
tomtomwombat/trailing
Oct 7, 2026
Merged

tomtomwombat merged 2 commits into
mainfrom
tomtomwombat/trailing

Conversation

@tomtomwombat

@tomtomwombat tomtomwombat commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Replaces old inline encoding. Trailing bytes of inline strings are 0b11111111, e.g. a len 3 string has 5 trailing u8::MAX. Then, len is calculated using leading/trailing ones:

let addr = self.addr();
    let trailing = if cfg!(target_endian = "little") {
    addr.leading_ones()
} else {
    addr.trailing_ones()
};
WIDTH - (trailing as usize >> 3)

This is simpler than previously, because the start position of utf-8 bytes are always 0 and the new encoding naturally works with 8 inlined bytes. Also, WORD_NUL_MAP is adjusted to (usize::MAX >> ALIGN_BITS * 2) << ALIGN_BITS;:

  • it's an impossible utf-8 representation, so doesn't collide with other inlined strings
  • it's symmetrical, so it's the same on big or little endian.
  • the first 2 leading bits are 0, classifying it as inlined (from TAG_MASK)
  • the trailing bits are 0, so leading/trailing 1s is 0, meaning it works with inline_len (although never used)

Inspired from @bonzini and https://github.com/bonzini/arcstring/blob/main/src/encoder.rs

@tomtomwombat
tomtomwombat force-pushed the tomtomwombat/trailing branch from 0632f55 to 65ef8aa Compare October 5, 2026 02:30
@tomtomwombat
tomtomwombat merged commit c99e235 into main Oct 7, 2026
1 check failed
@tomtomwombat
tomtomwombat deleted the tomtomwombat/trailing branch October 7, 2026 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant