fix: keep glued lexemes intact when formatting

[emrberk]

Problem

The formatter inserted a space between two word tokens that the lexer split from one QuestDB lexeme. The lexer never errors on these, so the "leave unknown text alone" path never ran.

Input	Output
0x1F	0 x1F
16777217.0f	16777217.0 f
geohash(5c)	geohash(5 c)
1500ns	1500n s

SELECT id, 0x1F FROM t formatted, ran green, and returned INT 0 in a column named x1F. Geohash DDL failed with "invalid GEOHASH size".

Fix

• spacing.ts: two word pieces with no whitespace between them in the source keep no gap. The dot rule treats any digit-led token as a number, so t. 5m no longer becomes the single token .5m.
• layout.ts: all phrase matching goes through one guard. A clause never starts on a word glued to the previous word, so 10.5mFROM no longer gets a newline inside it.

Two word tokens with no whitespace between them are never two lexemes in SQL. The rule closes hex, float suffixes, geohash precision, nanosecond durations, and future suffixes without listing literal shapes.

Tests

• oracles.ts: assertPreserved gains two oracles. Whitespace-stripped equality catches insertions. Main-lexer token equality catches merges. The existing stream oracle uses the formatter's own lexer, so it split 0x1F the same way on both sides and could not see this.
• lexemes.test.ts: eleven split lexemes in seven contexts, and an exhaustive sweep of 82 lexemes in every pair with three separators.

A random fuzzer over the same alphabet, 60,000 inputs on three seeds, reports zero character or lexeme changes after the fix. Before the fix it reported 353 extra failures.

🤖 Generated with Claude Code