How a Markdown escape should be represented in the editor

[Azganoth]

Question

A Markdown escape is consumed when a file is parsed and re-derived when it is saved, so between those points the editor holds no representation of one. Should it, and what should an author be able to see and do with an escape?

Context

#239 made a run of text that is exactly one link's source become that link when the caret leaves it. A file can hold those same characters escaped, deliberately literal, and the document cannot tell the two apart, so the engine now records for the current session which ranges it wrote and which loaded runs a change lands in, and refuses to commit the latter. That holds the escape where it can see the write, and two paths defeat it: a change that rebuilds a region re-inserts content it already held, which the step maps cannot tell from typing; and an in-app copy of a literal run reaches the clipboard unescaped (#244) and is pasted back as a run the session wrote, so the escape does not survive a copy round trip even after #244 is fixed. It also exposes three things the current model cannot express:

• The distinction is invisible. Two runs that read identically behave differently, and nothing on screen says which is which.
• Removing an escape means deleting the whole run and retyping it, because there is no character to delete.
• The clipboard carries characters, not their literalness. Its Markdown flavor is written unescaped for a text-only selection (Copying a text-only selection writes its Markdown unescaped #244), and with Commit typed link source to a link when the caret leaves it #239 in place, pasting an in-app copy of literal link source commits it to a link.

Nothing here is specific to links. The same holds for \#, \*, and every other escape the serializer writes, and for the corpus fixture under Escaped link syntax remains literal.

Related context

• Related issues or pull requests: Typed link source survives the save only when it is not followed by a space #234, Typed link source becomes a link #238, fix: keep typed link source literal when a space follows it #242, Commit typed link source to a link when the caret leaves it #239 and its pull request, and Copying a text-only selection writes its Markdown unescaped #244
• Documentation, decisions, or prior investigation: docs/specification.md Inline Content, docs/architecture.md Source Projection, and corpus/commonmark/links-and-images.md

Validate

• Where an escape could live: a mark applied at parse time, a session-scoped record in the projection engine as Commit typed link source to a link when the caret leaves it #239 does, or nothing durable at all.
• Whether source projection is the surface for it. Showing \[a]\(b) while the caret is inside would make the escape visible and let deleting the backslash convert the run, replacing today's delete-and-retype.
• What an author does to reverse an unwanted conversion, and whether Undo alone is enough once a document has been saved and reopened.
• What the clipboard should carry, and whether an in-app copy should preserve literalness that its Markdown flavor cannot express.
• What each option costs, measured rather than carried over. The scan in Commit typed link source to a link when the caret leaves it #239 costs roughly ten times more per keystroke on a 10 000-character paragraph holding a hint character near the caret than on one without, so the per-keystroke figure in that pull request holds for ordinary paragraphs and understates the tail. Measured under the vitest editor mount, so the ratio transfers and the absolute values do not.
• How each option interacts with projection, history, and copy.

Exit criteria

• The question is answered.
• The outcome below records the conclusion, tradeoffs, and supporting evidence links.
• Follow-up issues are created only for selected work.
• Owning documentation updates are identified when the outcome changes durable direction.

Outcome

Nothing durable is stored. The escape is derived where the caret meets it.

No mark, no session record, no new document state. An escape is recovered from the text and the serializer's safety check, both of which the document already holds, and shown as a two-character source projection — the backslash and the character it escapes — when the caret arrives. Deleting the backslash converts the run.

That second half is conditional on escaping becoming precise first, and does not stand before it. The sequencing is the whole answer, so it is set out below rather than left to the follow-up list.

The premise did not survive checking

The question assumes something is lost between parse and save. Measured against 9626471d, nothing is. Every escape form probed preserves the document across a save and reopen, including the fixture this issue names:

Document text	Saved	Document after reopen
[intentionally literal](garden.md)	\[intentionally literal]\(garden.md)	preserved
[a](b) from \[a\](b)	\[a]\(b)	preserved
*not emphasis*	\*not emphasis\*	preserved
# not a heading	\# not a heading	preserved
\ from a \\ b	a \ b	preserved

Fourteen forms, all preserved: \[, \], \*, \_, \`, \#, \!, \<, \-, \>, \|, \\, \~, and an escaped definition label. Compared as ProseMirror documents rather than bytes, so the comparison sees literalness rather than form.

This settles the first item the #135 comment left open. corpus/commonmark/links-and-images.md's \[intentionally literal](garden.md) does not round-trip byte-identically — the serializer adds \( — but it stays literal text. The escape survives; its authored shape does not.

Also unchecked there and worth stating: the serializer does not under-escape. The backslash rule is context-aware, dropping the escape only where the next character cannot form one (a \ b, C:\Users\me) and keeping it where it could (\\#, \\[, \\*, end of line). Over-escaping is its only defect.

Five corpus files change at the document level across one save — blocks.md and lists-and-blockquotes.md and task-lists.md in node attributes only, html.md in a hard break becoming an html node, tables.md in column count. None of it is escape-caused. That is a separate defect, recorded as follow-up below.

So the loss this issue was opened to fix does not exist on the file round trip. It exists only under #246, which converts a literal run into a link. The exposure is created by that feature, not by the format.

Where an escape could live

Nothing durable — selected. The measured problem is over-escaping, and over-escaping is a property of the text and its context, not of what the author typed. garden_sensor_name needs no escape because intraword _ is never emphasis; \[a](b) needs no \( because the \[ already stopped the link. A precise safety check answers both, and a mark answers neither. Costs nothing at runtime and touches neither projection, history, nor the clipboard. Gives up authored escape form: \[a\](b) re-saves as \[a](b). That is a one-time diff on first save, and docs/specification.md already declines to guarantee byte-for-byte round-tripping.

A parse-time mark — rejected, and cheaper than expected. Escape positions are fully recoverable from mdast alone, with no parser fork: walk node.value against source.slice(node.position.start.offset, node.position.end.offset), and a source \ followed by the next value character marks that character escaped. Prototyped and measured at 2.1 ms over a 71 000-character document against 261 ms to parse the same document — 0.8% of parse cost, once per open, nothing per keystroke.

It is rejected on what it buys, not what it costs. Three things:

• It is not total. 7 of 582 corpus text nodes (1.2%) cannot be aligned, because character references rewrite the text alongside escapes. Those nodes fall back to the blanket rule, so the mark cannot be the sole authority for emission.
• Mark-driven emission alone is unsound. A * typed fresh carries no mark; emitting it bare turns it into emphasis on the next open. The precise safety check is needed regardless, and the mark rides on top of it.
• Once the check is precise, the mark's remaining value is authored form — recovering \[a\](b) rather than \[a](b) — paid for with a mark that has to survive split, join, copy, and paste.

Reconsider it if authored form turns out to matter. It does not today.

The session-scoped record #246 introduces — rejected as the durable answer. It records whether this session wrote a range, which is not the same question as whether text is literal, and the difference is where its two known holes come from: a change that rebuilds a region re-inserts content indistinguishable from typing, and a copy round trip re-marks a loaded run as session-written. Those are not implementation gaps to close. It also carries the only per-keystroke cost of the three — roughly 2× on a paragraph holding link syntax and 2.2× on the first write into an untouched region, per that pull request's own table — and grows without a cap.

Whether projection is the surface

Two shapes were considered, and they do not share a fate.

Projecting the whole literal run — rejected. Showing \[a]\(b) across the run requires an adapter target spanning text that is not an object, with no rehydration and no owner. Generalized, it means projecting every text run that could be syntax, which is the blanket rule again, now visible and interactive.

Projecting the escape alone — selected, sequenced behind precise escaping. A two-character projection anchored to the escaped character, not to the run, needs no run-spanning target and no representation. It shows what the file will hold, derived from the same safety check that writes it.

It does not work against today's serializer, and the reason is worth recording so it is not re-litigated. A file holding \[a](b) saves as \[a]\(b), because the blanket rule escapes the ( as well. Measured — \[a\](b), \[a](b), and [a]\(b) all produce the identical document and all save as \[a]\(b). Projecting that, an author deleting the first backslash gets no observable change: the run stays literal because \( alone still breaks it. The gesture is spelled as one deletion but only completes in pairs, and its first half is silent.

Precise escaping dissolves this. One backslash is enough to keep [a](b) literal, and measurement confirms the same for \*emphasis*, \~~struck~~, and \`code` — each parses as literal text with a single leading escape. One escape, one deletion, immediate conversion.

Two properties survive the fix and are accepted rather than solved:

• The escape sits where the syntax needs breaking, not where the author wrote it. Keeping ![alt](x.png) literal cannot be done by escaping the ! — measured, \![alt](x.png) yields a literal ! followed by a live link, because breaking the image leaves the link beneath it. The only single escape that holds the whole run is on the [, so the file will hold !\[alt](x.png). Under a projection that shows what the file will hold this is honest: it shows where the escape is, and deleting it produces the image. The cost is a one-time move of authored form, which this outcome already accepts.
• Nested runs still pair. Text [![alt](i.png)](l.md) needs two escapes even minimally, one for the outer link and one for the image. Deleting one yields the image without the link — a partial conversion that is arguably correct rather than a no-op, and narrow enough not to block on.

The shape also completes a symmetry that already half-exists. A link projects its source today, and per docs/specification.md Inline Content an edit making that source invalid commits it as literal text, so typing a \ into projected link source should already turn a link literal. This is that gesture's inverse. Inferred from the specification rather than measured; confirm before building on it.

It does not answer the first complaint in this issue's Context. A projection entered by the caret tells an author what an escape is once they suspect one; it does not distinguish two identical-looking runs on sight. src/features/editor/plugins/markerPresentation.ts already decorates ranges without touching document text, and is the cheaper answer to visibility if that is wanted separately.

The remaining sub-questions

Reversal. With nothing stored, nothing needs reversing on the file round trip. Deleting a projected backslash is the forward gesture, and Undo reverses it within the session like any other document change. After a save and reopen there is nothing to undo, but there is also nothing to reverse — the escape is derived afresh from the text, so the same gesture is available again on the next open. That is the property the stored options cannot offer: a session record is gone by then, and a mark would have had to survive the save.

Under #246 as it stands the hazard is real and belongs to #246: a commit that happened and was saved is past what Undo reaches. The projection is the answer to that, by replacing a guess with a gesture.

The clipboard needs nothing beyond #244. Once a text-only copy runs through the save serializer, the plain-text flavor carries the escapes, and the in-app HTML flavor already preserves literalness because it carries no syntax to reparse. The claim in this issue's Context that the escape survives no copy round trip even after #244 holds only under #246's session record, which marks the pasted run as session-written. Without that record the copy round trip is sound.

Interaction with projection, history, and copy. Nothing stored means nothing to keep consistent across split, join, copy, or paste. The escape projection enters the existing adapter precedence in docs/architecture.md at character granularity, and two measured cases place it flush against a live target — \![alt](x.png) yields text:"!" beside a link, and \<https://example.com> yields text:"<" beside one — so precedence at that boundary is a design question for the implementing issue, not an open one for this spike.

How the #135 comment's framing resolves

That comment is right that the two questions meet at whether emission is document-driven or blanket, and right about the scale. The measurement does not favor the mark, though: every one of its four named causes is a precision defect in the safety check, fixable without knowing what the author typed. Making emission precise is not the same as making it document-driven, and only the first is needed — and precise emission is what the projection then reads from.

Method

No prototype branch. The probes were throwaway files under src/features/editor/tests/, run with vitest and deleted; nothing here is carried over from another measurement. To reproduce:

• Literalness. Mount corpus/ files and the escape cases with setupMilkdownEditorMount, take view.state.doc.toJSON(), serialize with getMarkdown(), mount the result, and compare the two documents rather than the two strings. Byte comparison alone is what made this look like a loss.
• Escape recovery and its cost. Parse with mdast-util-from-markdown plus the gfm extensions, collect text nodes, align each value against its source slice, and time the alignment against the parse on corpus/commonmark/blocks.md repeated 40 times, medians of seven runs.
• How many escapes a run needs. Mount each partial escape form and read the resulting document shape and its serialization, which is what shows one escape sufficing, the serializer writing two, and the escape relocating off the character the author escaped.

Residual uncertainty

• The precise safety check was not built. "Most of the 12 over-escapes are precision-fixable" is inferred from the four named causes and the escape probes, not measured against an implementation. The projection depends on it entirely.
• The caret-time cost of the projection is unmeasured. The safety check runs once per save today; driving the projection means answering "will the character beside the caret be escaped?" per caret move. That is the same category of per-keystroke expense this outcome rejects feat: commit typed link source to a link when the caret leaves it #246's record for, and it deserves the same measurement rather than an assumption in either direction. It is the largest open question here.
• The symmetry with typing \ into projected link source is read from the specification, not observed.
• The 1.2% unalignable rate comes from a corpus built to be syntax-dense. Ordinary prose was not sampled.
• Whether removing over-escaping leaves the five document-level differences unchanged was not verified. They look unrelated.
• Measured under the vitest editor mount, so ratios transfer and absolute values do not.

Follow-up

Filed, with the two escaping mechanisms separated after measurement and the document-level defects split apart once isolated:

1. The serializer escapes characters that cannot be syntax where they sit #252 — the serializer escapes characters that cannot be syntax where they sit. Sub-issue of Preserve the authored form of a Markdown file on save #251, which had been holding this slot for this spike. Largest measured win, and a hard prerequisite for Project an escape as its two characters so deleting the backslash converts the run #255. Tightens test: assert corpus round-trip convergence for supported syntax #249 toward byte identity.
2. A bare URL is escaped on save with an escape that does not hold #241 — widened rather than duplicated. Two of this spike's escaping cases, name@example → name\@example and www.example_.com → www\.example\_.com, come from mdast-util-gfm-autolink-literal rather than from core safety patterns, which is the same mechanism A bare URL is escaped on save with an escape that does not hold #241 already owns for \:. The control that separates them: sensor.reading_value keeps its . bare and escapes only its _, so \. is emitted for the www. guard specifically. The serializer escapes characters that cannot be syntax where they sit #252 is scoped to exclude all three.
3. Copying a text-only selection writes its Markdown unescaped #244 — already specified; needs nothing from this spike.
4. A line break before an HTML block is lost on save #253 and Saving pads a ragged table row into new cells #254 — the two document-level defects, filed separately. A line break before an HTML block is lost on save; saving pads a ragged table row into new cells. These were one follow-up item until isolation showed two unrelated causes. Neither is escape-related, and neither is one of Determine whether the Markdown corpus can drive automated round-trip tests #135's three identity-gap causes: both converge cleanly, so corpusRoundTrip.test.tsx is green on them. They surfaced only because this spike compared documents rather than serialized bytes, which is the technique worth carrying into their regression coverage. Filed standalone rather than under Preserve the authored form of a Markdown file on save #251, following A list item's non-paragraph first child loses its nesting on save #247.
5. Project an escape as its two characters so deleting the backslash converts the run #255 — project an escape as its two characters. Sub-issue of Typed link source becomes a link #238, whose Scope already covers reverting an unwanted conversion. Blocked on The serializer escapes characters that cannot be syntax where they sit #252 and does not work before it. Typed link source becomes a link #238 also records that a bare URL has no literal state, so that form is out of scope there.
6. Commit typed link source to a link when the caret leaves it #239 — whether to narrow its trigger or wait for Project an escape as its two characters so deleting the backslash converts the run #255 is recorded in a comment on Typed link source becomes a link #238 rather than as an issue, since it is a direction change on open work rather than new work.

Owning documentation: no durable direction changes yet, so none is required now. docs/specification.md line 39 already declines byte-for-byte round-tripping. Item 5 does change durable behavior and owns a specification update under Inline Content when it lands; until then one sentence there stating that literalness is preserved by escaping on save and is not represented in the document would keep this question from being reopened.

[Azganoth]

Measurement from #135 that belongs here as an input, since it lands on the serializer side of the same question.

That spike ran serialize(parse(source)) over the 16 corpus files under commonmark/, gfm/, and isolated/end-of-file/. 12 of the 16 come back rewritten, and the largest single cause is the serializer emitting escapes that no author wrote and that are not needed where they sit:

• garden_sensor_name → garden\_sensor\_name — intraword _ is never emphasis in CommonMark
• *opening-only asterisk emphasis → \*opening-only asterisk emphasis — an unpaired * is already literal
• name@example → name\@example
• www.example_.com → www\.example\_.com

This is the inverse of the direction this issue starts from, and worth stating so the two are not conflated. This issue asks how an escape the author wrote stays distinguishable and survives editing and copying. The measurement is about escapes nobody wrote being emitted on save. Neither subsumes the other.

They meet at one decision, which is why this is an input rather than a separate concern: whether escape emission is driven by what the document records, or by a blanket "escape anything that could be syntax" rule. Today it is the blanket rule, which is what produces the output above. "A mark applied at parse time", the first option under Validate, is the option that makes mark-driven emission possible — the serializer would emit an escape where a mark records one and leave the character bare otherwise. The session-scoped record in the projection engine does not obviously extend that far, and "nothing durable at all" leaves the blanket rule in place by construction. So the measurement is not neutral between the three options, and that seemed worth surfacing before they are weighed.

It also sets the scale. Over-escaping is not a cosmetic tail case: it affects most of the supported-syntax corpus, so whichever option is chosen, "does the serializer escape correctly" is part of the cost, not a detail deferred past it.

Two things deliberately not established, so they are not read into the above:

• Whether the escaped-link fixture round-trips. corpus/commonmark/links-and-images.md is one of the 12 that differ, and it carries the \[intentionally literal](garden.md) case under "Escaped link syntax remains literal" that this issue names. The run compared whole files and did not isolate that fixture, so whether that escape survives is unchecked — and it is precisely this issue's subject.
• Whether any of this is semantically lossy. The samples inspected changed the text while preserving meaning. That was not checked across all 12 files and is not claimed.

Full evidence, method, and the three causes of non-identity are in the Outcome section of #135. The convergence guard from that spike is #249; it asserts stability, not identity, precisely because this question is open, and is written to tighten toward byte identity once it is settled.