Skip to content

encode() base64-encodes every string on one line, so .lino files are not human-readable — the readable formatter already exists but is not the default #37

Description

@konard

Summary

encode() base64-encodes every string and emits the whole object on one line. The result is not human-readable, which defeats the point of Links Notation as "the portable, human-readable syntax" and "the reviewable import/export projection".

Requested: make an indented, plain-text form the default, and keep the current single-line encoded form available under an explicit name.

What is produced today

Real stored file from a deployment that persists its state through this crate:

(object ((str dHlwZQ==) (str Um91dGVyU3RhdGU=)) ((str c3VidHlwZQ==) (str VG9rZW5TdG9yZQ==)) ((str dm…

Decoding by hand: dHlwZQ==type, Um91dGVyU3RhdGU=RouterState, c3VidHlwZQ==subtype, VG9rZW5TdG9yZQ==TokenStore.

The structure is sensible, but nobody can see that without running base64 on every field. Reviewing a diff, grepping for a key, or spotting a wrong value are all impossible.

Encoding is unconditional (src/lib.rs:33), so ASCII keys like type need it as much as text that actually does. src/lib.rs:14 states the reason:

UTF-8 Support: Full Unicode string support using base64 encoding

Unicode support is a real requirement, but the notation already has a quoting strategy for it, and this crate implements it in format_indented_value — single quotes, double quotes, and doubling for embedded quotes.

The proposed default

One construct — ( ) — for both objects and arrays, at every level including the root. Indentation is for readability only; what distinguishes the two is the content: key value pairs make an object, bare values make an array.

JSON:

{
  "type": "RouterState",
  "server": { "host": "127.0.0.1", "port": 18878 },
  "models": ["claude-haiku", "claude-opus"],
  "value": [
    { "id": "7cf7abf6", "label": "bootstrap-admin", "ttl_hours": 24, "revoked": false },
    { "id": "94b36f7e", "label": "wrapper-run", "ttl_hours": 720, "revoked": false }
  ]
}

Links Notation:

(
  type "RouterState"
  server (
    host "127.0.0.1"
    port 18878
  )
  models (
    "claude-haiku"
    "claude-opus"
  )
  value (
    (
      id "7cf7abf6"
      label "bootstrap-admin"
      ttl_hours 24
      revoked false
    )
    (
      id "94b36f7e"
      label "wrapper-run"
      ttl_hours 720
      revoked false
    )
  )
)

Mapping

JSON Links Notation
object, including the root ( ) containing key value pairs, one per line
array ( ) containing values, one per line
object as an array element its own ( )
string double quotes, no encoding
number, true/false, null bare, so the type survives the round trip

Why parentheses rather than indentation alone for a nested object

Both parse, but they mean different things. Verified against the parser:

server
  host "127.0.0.1"
  port 18878

(server (host …)), (server (port …)) — two separate facts about server.

server (
  host "127.0.0.1"
  port 18878
)

server → [(host …), (port …)] — one object with two fields, which is what the JSON meant.

The same argument applies to array elements: without their own ( ), the fields of consecutive records merge into one flat list and the record boundaries cannot be recovered.

Dependency

This form relies on parentheses opening a nested indentation context, which landed in link-foundation/links-notation#283 and is released as links-notation 0.14.0.

This crate currently depends on links-notation 0.13, where the behaviour is absent: inside ( ), indentation is ignored and every line collapses into one flat list. So the dependency has to be raised to 0.14 before the new default can work.

Verified against the 0.14 parser: the document above parses into exactly the structure described, and

value (
  id "1"
  label "one"
)

now yields two links — (id "1") and (label "one") — where 0.13 produced four loose references.

What is being asked

  1. Raise the links-notation dependency to 0.14, so parentheses open a nested indentation context.
  2. Make the indented, plain-text form above the default output of encode().
  3. Keep the current single-line base64 form as a configurable option under an explicit name — encode_compact() or encode_obfuscated() — so the code and its tests are preserved and callers opt into it deliberately. Any other output styles stay available the same way: configurable, never the default.
  4. Encode a value only when it genuinely cannot be represented as text, and mark those values individually rather than encoding everything.
  5. Accept both forms when decoding, so existing files keep working and migrate on next write.
  6. Update the docs: src/lib.rs:866 currently teaches the opposite — // String "Alice" is base64-encoded as "QWxpY2U=".

The point is the default. A library called without reading its documentation should produce a readable file; today the readable form exists but is reachable only by knowing about a separate module.

Tests worth adding

  • encode() of an object with ASCII keys and values produces output containing them verbatim;
  • output spans multiple lines with indentation, not one line;
  • nested objects and arrays round-trip to the same structure;
  • an object used as an array element keeps its boundary;
  • numbers and booleans survive the round trip as numbers and booleans, not strings;
  • a value containing quotes, newlines or non-ASCII text round-trips exactly;
  • a value that cannot be represented as text is marked individually and round-trips;
  • a file written in the previous base64 form still decodes.

Note

Found while auditing what a downstream project stores on disk. Every fact needed to read the file was present, but only after base64-decoding four fields by hand to discover the structure was correct all along.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions