Skip to content

The serializer escapes characters that cannot be syntax where they sit #252

Description

@Azganoth

Summary

Saving rewrites text that needed no escape. The serializer asks whether a character could be Markdown syntax rather than whether it still can be where it sits, so it emits escapes no author wrote. Out-of-context escaping is the largest single cause of the identity gap #135 measured, and the rewrites are what stop #249 from asserting byte identity.

The fix is to make the safety check precise. Escapes the document genuinely needs must keep being emitted; this issue removes escaping that changes nothing, not escaping.

Steps to reproduce

  1. Open a document containing garden_sensor_name.
  2. Save it.
  3. Read the file.

Expected behavior

garden_sensor_name. Intraword _ is never emphasis in CommonMark, so no escape is needed.

Text that would parse differently without an escape keeps it. Both directions matter: the current rule never under-escapes, and that must remain true.

Actual behavior

Measured against 9626471d by driving the editor mount used by the plugin tests:

Document text Saved
garden_sensor_name garden\_sensor\_name
sensor.reading_value sensor.reading\_value
*opening-only asterisk emphasis \*opening-only asterisk emphasis
text with [ bracket text with \[ bracket
[intentionally literal](garden.md) kept literal \[intentionally literal]\(garden.md)

The last row is the case #245 turned on. One escape is enough to keep that run literal, and the serializer writes two: \[a](b), [a]\(b), and \[a\](b) all produce the identical document, and all save as \[a]\(b).

12 of the 16 corpus files under commonmark/, gfm/, and isolated/end-of-file/ come back rewritten, and this is the largest single cause.

Related context

Done when

  • The cases above save without an escape they do not need.
  • Escaping still holds for every form that needs it, including the fourteen escape forms How a Markdown escape should be represented in the editor #245 verified and the context-aware backslash cases (a \ b, C:\Users\me, \\#, \\[).
  • Regression coverage asserts both directions, so a later change cannot buy precision by under-escaping.
  • corpus/commonmark/links-and-images.md round-trips byte-identically, or the reason it cannot is recorded.

Notes, logs, screenshots

Diagnosis

serializeMarkdownText in src/features/editor/utils/markdownText.ts delegates to remark-stringify's state.safe, which escapes on the possibility of syntax rather than on its presence. The handler is already overridden for trailing whitespace, so the seam exists.

The core rule is already partly context-aware, which narrows the fix to specific patterns rather than a wholesale replacement: text with ] bracket is not escaped while text with [ bracket is, and the backslash rule correctly drops the escape in a \ b and C:\Users\me while keeping it in \\# and \\[.

Implementation direction

The check needs the character's context, not the document's history. #245 established that no representation of an escape is needed or wanted for this: every case above is decidable from the text around the character.

Out of scope

Metadata

Metadata

Assignees

Labels

BugSomething isn't working

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions