A character reference is decoded and saved as the literal character

[Azganoth]

Summary

A character reference is decoded when a file is opened and written back as the literal character it names, in text and in link destinations alike. The rendered document is unchanged, so this is an authored-form rewrite, but it is one an author is likely to notice: a file written to stay ASCII does not stay ASCII.

Steps to reproduce

1. Open a document containing © © © Æ.
2. Save it.
3. Read the file.

Expected behavior

© © © Æ. The reference and the character it names render identically, so nothing requires rewriting one into the other.

Actual behavior

Measured against 9626471d by driving the editor mount used by the plugin tests. Stable across a second save:

Document	Saved
© © © Æ �	© © © Æ followed by U+FFFD
[Entity-obfuscated scheme](javascript:alert(1))	[Entity-obfuscated scheme](javascript:alert\(1\))

A malformed reference is left alone and gains an escape instead: &copy &MadeUpEntity; &#; &#x; saves as \&copy \&MadeUpEntity; \&#; \&#x;.

The second row matters beyond form. A destination written to disguise its scheme is saved in the open, which changes what a reader and any other tool see.

Related context

• Related issues or pull requests: Preserve the authored form of a Markdown file on save #251 is the parent; How a Markdown escape should be represented in the editor #245 answered the neighbouring question of whether an escape needs a representation in the document and concluded that nothing durable should be stored; The serializer escapes characters that cannot be syntax where they sit #252 fixed the escaping defect measured in the same pass; test: assert corpus round-trip convergence for supported syntax #249 owns the corpus guard.
• Documentation, decisions, or prior investigation: corpus/commonmark/text-and-breaks.md under the character-reference section, and corpus/commonmark/links-and-images.md under Script-like and data schemes are ordinary destinations.

Done when

• A well-formed character reference saves as it was authored, or the decision to keep decoding it is recorded with its rationale.
• U+0000 continues to be written as U+FFFD, which CommonMark requires.
• A malformed reference keeps behaving as it does today, as literal text.
• Regression coverage asserts the saved form.
• The owning documentation records the accepted answer, since it decides what a saved file may differ by.

Notes, logs, screenshots

Diagnosis

micromark decodes character references while parsing, before the mdast text node exists, so by the time anything in this repository sees the value the reference is gone. serializeMarkdownText in src/features/editor/utils/markdownText.ts receives the decoded character and has nothing to distinguish it from one the author typed directly.

Implementation direction

Deliberately open. #245 examined the same shape of question for escapes and rejected both a parse-time mark and a session-scoped record, but it did so because escaping is derivable from the text and its context, which a character reference is not: © and © are indistinguishable after the parse. Whichever way this goes, it decides whether #251's byte-identity target admits an exception, so it is worth settling before the remaining sub-issues rather than after.

#245 also measured the cost of the alignment walk this would need, at 2.1 ms over a 71 000-character document against 261 ms to parse it, and found that 7 of 582 corpus text nodes could not be aligned — because character references rewrite the text alongside escapes. That measurement was taken for the escape question and applies directly here.

Out of scope

• Escapes, which The serializer escapes characters that cannot be syntax where they sit #252 and How a Markdown escape should be represented in the editor #245 own.
• The link title and destination normalizations measured in the same pass, which are filed on their own.