fix: encode mint, multiasset and withdrawal keys in CIP-21 canonical order

[ptrdsh]

Problem

Mint, MultiAsset and Withdrawals encode their CBOR maps in insertion order. CIP-21 requires policy IDs, asset names and withdrawal reward accounts in canonical CBOR key order (RFC 7049 §3.9: shorter first, then bytewise).

Hardware wallets never receive the transaction bytes. They get the body field by field, serialize it themselves in canonical order, and sign the hash of their own serialization. When the SDK emits keys in another order, the device's body hash doesn't match the transaction, and the signature is useless.

This came up with a real preprod deployment built with the SDK: the tx minted under three policies, inserted in the order 56ed…, 8cba…, 7379…. cardano-hw-cli transaction validate reported CBOR is not canonical, and the Ledger could not sign it.

Change

• Bytes.compareCanonical: the canonical byte-string key order.
• Mint.FromCDDL, MultiAsset.FromCDDL and Withdrawals.FromCDDL sort entries with it on encode. Decode is unchanged.

Decoded transactions keep their bytes. The format-preserving path (Transaction.fromCBORHex → toCBORHex) replays each map's recorded keyOrder (CBOR.ts, the map-encoding branch). A transaction decoded from a provider or wallet therefore re-encodes byte-for-byte, so its hash and existing signatures stay valid. Only freshly built values get the canonical order.

No hashes are affected. The script data hash covers the redeemers, datums and language views, not the body. Mint redeemer indices were already derived from sorted policy IDs (txBuilder.ts).

Tests

test/CIP21.mapKeyOrder.test.ts:

• Mint and MultiAsset with policies in the real-world order above, and asset names ff, 0000, 01. These must come out as 01, ff, 0000: length decides before byte value.
• Withdrawals with reward accounts inserted in reverse order.
• A decoded transaction with a non-canonical mint map round-trips byte-for-byte.

With the src/ change reverted, the three ordering tests fail and the round-trip test passes. With it, all four pass. The full packages/evolution suite passes: 1260 passed, 75 skipped. tsc -b tsconfig.src.json is clean.

Not in this PR

CANONICAL_OPTIONS is not a fix for this. Re-encoding a Plutus transaction with it rewrites the redeemer encoding, but the body keeps the script data hash the builder computed under the default options, so the result fails ledger validation. Details: #585.

[jorbuedo]

Strong PR, and the write-up with the cardano-hw-cli repro is the right kind of evidence. Sorting inside FromCDDL.encode rather than reaching for sortMapKeys looks correct to us: it keeps plutus_data out of scope, so datum hashes are untouched, and Bytes.compareCanonical matches RFC 7049 §3.9. (And #585 is a good catch, independently.)

We build a mobile Cardano wallet and maintain a differential harness that runs an encoder against CSL 15.0.3, CML 6.2.0 and two cores of our own, comparing bytes on construction rather than round-tripping. We ran 059962a7 through it. Mint, MultiAsset and Withdrawals now come out byte-identical to CSL when encoded directly.

The catch is that four encode paths build the same byte-keyed maps inline instead of delegating to the modules this PR fixes, and three of them are on the transaction path.

file	encodes	reached by	delegates?
Value.ts:305	multi-asset inside a transaction output	every token send	no, inline copy of MultiAsset.FromCDDL
Assets.ts:663	the SDK-facing Assets CDDL encode	payToAddress	no, inline copy
TransactionBody.ts:253	body key 5, withdrawals	every withdrawal tx	no, does not use Withdrawals.FromCDDL
GovernanceAction.ts:454	treasury withdrawals map	treasury proposals	no, inline

Mint is the one that does delegate: TransactionBody.ts:184 is ParseResult.encodeEither(Mint.FromCDDL).

That is probably why the repro in your PR description validated after the fix. The failing transaction you describe was a three-policy mint, which is exactly the field that goes through Mint.FromCDDL. An equivalent transaction carrying those assets in an output instead would still fail cardano-hw-cli transaction validate on this branch.

Reproduction

const P1 = PolicyId.fromHex("11".repeat(28)), P0 = PolicyId.fromHex("00".repeat(28))
let ma = MultiAsset.singleton(P1, AssetName.fromHex("aabbcc"), 5n)
ma = MultiAsset.addAsset(ma, P1, AssetName.fromHex("ff"), 7n)
ma = MultiAsset.addAsset(ma, P0, AssetName.fromHex("ff"), 9n)

MultiAsset.toCBORHex(ma)                        // fixed by this PR
Value.toCBORHex(Value.withAssets(1234567n, ma)) // still insertion order

MultiAsset.toCBORHex   a2 581c00…00 a141ff09 581c11…11 a241ff0743aabbcc05        <- matches CSL
Value.toCBORHex    821a0012d687 a2 581c11…11 a243aabbcc0541ff07 581c00…00 a141ff09
CSL 15.0.3         821a0012d687 a2 581c00…00 a141ff09 581c11…11 a241ff0743aabbcc05

A Babbage output built on this branch, same assets:

this branch   a200581d6052e63f…01 821a0012d687 a2 581c11…11 a243aabbcc0541ff07 581c00…00 a141ff09
CSL 15.0.3      82 581d6052e63f…   821a0012d687 a2 581c00…00 a141ff09 581c11…11 a241ff0743aabbcc05

The a2-map vs 82-array framing there is a separate matter (#577); only the value payload is at issue.

Suggestion

Have Value.ts, Assets.ts, TransactionBody.ts (withdrawals) and GovernanceAction.ts delegate to the FromCDDL encoders this PR fixes, rather than adding a fifth sort. The duplication is what let them diverge.

On the tests

CIP21.mapKeyOrder.test.ts asserts at module level on Mint, MultiAsset and Withdrawals, which is why all four tests pass while the output path is unchanged. A case at TransactionOutput or TransactionBody level would have caught it.

The general point, which is what we would most like to contribute: a round-trip property test is structurally blind to this class of bug, because decode preserves whatever order the input had, so encode∘decode is order-neutral no matter what the encoder does. Only construct-from-scratch-then-compare-against-a-reference finds a construction ordering defect. Three of the *.CML.test.ts files (Mint, Withdrawals, GovernanceAction) are round-trip only and import no oracle, and there is no MultiAsset.CML.test.ts at all, which is the same gap this bug lived in.

What we plan to do

We are preparing a few PRs against the SDK and would like to pick up this rework: the four delegations above, plus construct-then-compare oracle tests at TransactionOutput and TransactionBody level so it stays fixed. Happy to do that as a follow-up to this PR, or to hand you the failing vectors if you would rather fold it in here. Just say which you prefer.

Two limits on the above, so nothing reads as more than it is. We have not run your suite on this branch, so we make no regression claim, only a statement about what the new test covers. And we have not put a transaction from this branch in front of a device ourselves. The ordering results are byte comparison against CSL; the hardware consequence is yours, from your repro, not something we re-observed.

[ptrdsh]

thx for the quick reply.
all good points and I am happy if you follow it up to this PR, as suggested. Its an easy local fix once known. No rush therefore.

[solidsnakedev]

Thanks for the careful write-up and the real-world repro; this is exactly the kind of evidence that helps.

We are keeping the default encoding as it is, so we won't merge a change that reorders freshly built maps. Hardware-wallet signing will go through an opt-in canonical mode instead, which needs no per-module sorting: CBOR.CANONICAL_OPTIONS already writes these maps in canonical order. I checked it against cardano-hw-interop-lib 3.1.0. On a transaction with unsorted mint policies, tokens ff, 0000, 01 in the mint and in an output, a script and a key withdrawal, an indefinite datum and unsorted metadata labels, the SDK's canonical bytes are byte-identical to the library's own re-encoding, and validateTx reports no issues.

What is missing is the builder side, which you described in #585: the script data hash, metadata hash and fee are computed with the default encoding. The plan there is #579 first, then #581 so a built transaction keeps the bytes it was built with, then an encoding option on build().

Closing this in favour of #585. Your test vectors (the three-policy order and the ff, 0000, 01 names) would be very welcome as tests for the canonical mode there.