Repository navigation
Diff, List, Base64 and JSON upgrades; offline; JSON to Types - #24
Merged
Merged
Conversation
- C-8: drop the unneeded mode dependency in MergePane; lint is clean. - ES worker output, so a worker's dynamic import stays a separate chunk (Prettier, sql-formatter). - C-5: vite-plugin-pwa precaches every built asset; external registerSW.js, no manifest. - C-6: one static 1200x630 og.png (source in scripts/og-card.html), og:image and twitter:image on every page. - C-7: src/lib/registry/about.ts, read only by scripts/seo.ts, typed Record<ToolSlug, string>. - Core share subpath (encodeCompact / decodeCompact), with diff share links rebuilt on it. - JSON tool persists to IndexedDB (stateVersion 2); dompurify pinned for the SVG inspector. - ColorField and Swatch move to components/ui.
Replaces the 27 TODO placeholders in src/lib/registry/about.ts with one paragraph per tool, rendered only into the prerendered HTML by scripts/seo.ts. Each paragraph names edge cases the tool actually handles, and every claim was checked against that tool's engine and tests in packages/core, its shell, or its registry store kind. Features still in flight on other branches are not described.
A "Copy link" button under the steps copies a link that carries the pipeline's steps (and any set-operation list) but never the input text. It lives in the URL fragment, not a query string, so it is never sent to the server or its logs. The link is encoded ahead of the click so the clipboard write happens directly in the handler, as Safari requires; past 8000 encoded characters, or when a setting is out of the shareable bounds, the button is disabled and the visible hint says why.
Opening a link (on load, or when pasted into an already-open List tab) replaces the steps, keeps the input, clears the fragment while keeping React Router's history state, and shows a dismissible notice. A damaged or newer-version link shows an error and changes nothing. On /list/<kind> the tool moves to /list so the route kind cannot swap a loaded one-step pipeline.
Core: list/share.ts adds encodePipeline, decodePipeline and a strict parseOp/parseOps/parseSharedSteps that rebuilds every op field by field (types, enums, string lengths, the LineTest union, and bounds on pad, chunk size and seed), drops unknown fields, and rejects the whole link on any bad value. Payload is versioned ({ v: 1, steps: [{ op, on }] }) inside the codec's own 1. prefix. Unit tests per kind plus property tests: valid pipelines round-trip, arbitrary JSON never makes the parser throw, arbitrary strings never make decodePipeline throw.
Also clamps the filter step's line-number range loop to the document, so a range like 1-999999999999 no longer walks every number in it.
… before CodeMirror sees it The size guard now runs on the live text instead of a deferred copy, so a 5 MB single-line file that is picked, dropped, restored from IndexedDB or opened from a share link is refused in the same render and never reaches CodeMirror. assessSize scans lines with indexOf (5 MB single line: 14.8 ms to 0.1 ms) and its thresholds are exported as MAX_DIFF_CHARS, MAX_DIFF_LINE_CHARS and LINE_ONLY_CHARS. MergePane gets a transaction filter that refuses an edit (typically a paste) that would push a side past those limits, with a notice, so the editor never lays out the oversize line first. MergePane no longer echoes text pushed in from React back through the change callbacks, tracks the last text per side by reference instead of comparing doc.toString() on every change (which also stops CRLF input from being re-pushed into the other pane on each keystroke), keeps the unified view's original document in sync with the left side, and accepts a readOnly prop. New core transformForDiff(text, 'sort-lines' | 'json'): sort-lines uses the List tool's natural sort with a code-point tie-break, so the output depends only on the multiset of lines, and keeps CRLF, BOM and the final newline; json accepts comments and trailing commas, deep-sorts keys, prints with two-space indent, keeps big integers losslessly where JSON.rawJSON exists and otherwise says how many numbers were rounded, and reports invalid input with a line and column. The Diff tool gets a "Compare as" select (As written / Sorted lines / Key-sorted JSON). While a transform is active both panes show the rewritten text read-only under a persistent banner with a Show original button and per-side JSON errors; stats and the copied patch use the shown text and the button says so; the share link keeps the original text; the size guard applies to the rewritten text.
… with SHA-256 A decoded SVG is valid UTF-8, so it used to show only as markup. Decode mode now previews it through <img> above the text, and every SVG (encode-mode file, decoded text, or a decoded non-UTF-8 SVG) gets an inspect panel: pixel size and viewBox, plus a strip report from running DOMPurify (svg + svgFilters profiles) on the source text, e.g. "Stripped 1 <script>, 2 on* handlers, 1 javascript: URL". Copy and Download give the sanitised SVG as text only; it never enters the page's DOM. DOMPurify is imported on first inspection and lands in its own chunk. Inputs above 1 MB say "Too large to inspect". Non-UTF-8 decodes now open in a virtualised hex view instead of a 4 KB plain-text dump, with a Hex / Text (legacy encoding) toggle, the sniffed type, size and SHA-256. Encode-mode files get the same view under a new Encoded / Hex toggle, up to the 25 MB file cap. Only visible rows are formatted; inputs over 8 MB are paged so the scroller stays under browser height limits. Copy takes the dump of the first 1 MB; Download builds the whole dump a chunk at a time into a Blob, yielding between chunks. Core: hexRow, formatHexRow, hexDumpRows and hexRowCount share one row formatter with hexDump; summarizeRemovals groups normalised sanitiser removals into a report; standaloneSvg restores the namespace declarations an HTML serialiser drops; svgRoot reads the root's raw width, height and viewBox.
convert() ran synchronously on the main thread over a deferred copy of the state, so a large paste still blocked the page for the whole parse and re-serialise. It now runs in yaml-toml.worker.ts through useWorkerJob, with the usual "still converting after 2s" Cancel; errors with line and column, warnings, notes, the document picker and the detected format all come from the worker's result as before. yaml, smol-toml and jsonc-parser moved with it: the YamlTomlTool route chunk drops from 42,954 to 2,839 bytes gzip, and the libraries now live only in the worker chunk. The input's byte count no longer builds a Blob of the whole text on every render. It is a surrogate-aware UTF-8 length loop over the deferred text.
The converter could write shell export lines but not read them. EnvInputFormat gains 'shell', read by parseShellExports, and the tool's input picker gains "Input: shell export". Auto-detection picks shell only when dotenv reports an error, the shell reader reads the text without one, and the text has an export line. An ordinary .env file with an export prefix stays .env: where both readers accept a file they either agree or (for "a\nb" in double quotes) disagree in a way dotenv users expect dotenv to win. What tips it to shell is quoting only sh reads, such as 'it'\''s' and A='x'; B='y'. Detection parses once and convertEnv reuses that parse. parseShellExports now reads several commands on a line separated by `;`, as sh does. convertEnv strips a BOM and turns CRLF into LF before the shell reader sees the text, matching the .env reader; a lone CR inside quotes is still kept. Tests cover the detection boundary in both directions, `;`-separated commands, CRLF and BOM input, and a fast-check round trip from toShell output (LF and CRLF) through convertEnv.
A 3 MB single-line paste cost 3.5 to 5.7 s from keystroke to next paint. Measured in Chrome on the built site, with edits made through execCommand('insertText') so they take the real editing path, two causes add up.
The bigger one is the browser. Chrome re-lays out the whole paragraph on every edit, and a soft-wrapped 3 MB line took 2.2 to 8 s per keystroke in a bare textarea with no React and no JS of ours. The same textarea with wrap="off" took 0.2 to 0.8 s, and the same 3 MB split into 100-character lines took about 0.1 s. The input now turns soft wrap off once any line passes 100,000 characters. hasLongLine checks with one indexOf per line, on the keystroke itself, so a paste lays out unwrapped the first time. The textarea scrolls sideways natively, touch included. CodeArea needed no new prop because it already passes `wrap` through; its doc comment now describes the trade.
The smaller one is ours. The encode ran in a useMemo over a deferred value, but React cannot interrupt one synchronous computation, so every keystroke still paid 150 to 300 ms of encode plus GC of a 27 MB binary string. Encode and decode now run in text-bytes.worker.ts through useWorkerJob, with the usual Cancel after 2 s. Swap feeds the output back only when it belongs to the current input.
After both changes, the same paste costs 0.19 to 0.21 s from keystroke to paint, which is about what the bare unwrapped textarea costs.
Known limit: Chrome caps a box's scroll width at 16,777,216 px. With wrap off, characters past about 2.2M on one line (13px desktop font; about 1.7M at the 16px mobile font) are still in the value, but they can't be scrolled into view. Typing, select-all, Copy and Download still cover all of it.
- New core subpath /json-types with a hand-written inference engine, no new runtime dependency. Every sample is merged into one shape; a missing key is optional and a null is nullable, kept apart in all three outputs; ints and floats are told apart by literal via parseLossless; integer width is tracked through int64 and uint64. - Identical structures share one named type; name collisions get the parent prefix, then a number. Go uses golint initialisms, gofmt column alignment, pointers and omitempty for optional fields, and reports keys a struct tag cannot hold. - Zod output uses syntax valid in v3 and v4, children first; .int() is dropped past 2^53 because Zod 4 rejects it there. - Tests check that the emitted Zod schema parses its own sample (fast-check), that tsc accepts each sample against its emitted TypeScript, and, where Go is installed, that gofmt is clean and every sample decodes with DisallowUnknownFields. zod is a dev-only dependency of core. - App: src/tools/json-types runs the engine in a worker via useWorkerJob (28 MB re-runs in ~2s); IssueCard moves to components/ui for its third user.
Above 1 MB the tool validates and formats only: auto-fixes, embedded-JSON expansion and lossless re-parsing are off, with a note that stays visible. Above 50 MB nothing is parsed. The byte count is a UTF-8 length scan on the deferred text instead of a Blob per keystroke. Output over 2 MB shows its first 100k characters, with Copy and Download for the rest. At most 100 problems are listed, error snippets cut a long line to a window around the caret, stripJsonc copies in slices, and the tree caps rows and node counting. Measured on 20 MB of NDJSON in Node: size 13 ms, analyze 125 ms, number scan 51 ms, wrap as array 11 ms, format 53 ms.
Three tools had their own copy of the long-line check and two their own UTF-8 byte counter; both now live in @fadeltd/devtools-core/text with tests. The JSON input stops soft-wrapping past a 100k-character line, as Text to Binary / Hex and JSON to Types already do, so a pasted minified file stays editable. The large-input note now also says editing may feel slow and where parsing stops.
fadeltd
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added
Changed
exportfilesFixed