Repository navigation
Add UPLC → JVM bytecode demo built from eo optics - #132
Merged
Merged
Conversation
A compiler from Untyped Plutus Core (Cardano smart contracts) to JVM class
files whose spine is a single polymorphic optic:
Iso[Array[Byte], Array[Byte], TypedProgram, JvmProgram]
S: flat UPLC T: class file A: typed expr B: JVM-shaped IR
`get` decodes and types, `reverseGet` emits bytecode (ASM), and the compiler
is `iso.modify(Lower.lower)`; `cborScript andThen iso` takes on-chain CBOR.
- Typer (S ⇒ A): re-sugars let / if / saturated builtin calls / direct calls
of let-bound functions and forced builtins / case-on-constr multi-argument
calls, with rewrites exact under CEK evaluation order. Hybrid typing keeps
provable integers, booleans, bytestrings and strings unboxed.
- Lower (A ⇒ B): closure conversion with a capture layout fixed by the
source, slot allocation, explicit boxing, typed fast paths, and let-body
outlining to stay under the JVM's 64 KB method limit.
- Runtime: closures, partial builtins, PV11 case-on-builtins, discharge back
to UPLC terms; builtins delegate to Scalus except the five selectors.
Correctness is differential against the Scalus 1.3.0 CEK machine (Plutus V3,
PV11, variant E): the vendored Plutus conformance suite (plutus 1.63.0.0,
943/943 evaluation cases agree) and Scalus-plugin-compiled validators with
accepting and rejecting V3 script contexts. Every generated class passes
ASM's verifier. Results and errors are compared; budgets are not metered.
The demo and its validator corpus live outside the root aggregate; CI runs
`demoUplcJvm/test` as its own step and checks their formatting and scalafix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
Contributor
|
🚀 Cloudflare Pages preview for https://0927a1f3.cats-eo-docs.pages.dev Branch alias: https://ccr-1ec1a317-uds4rn.cats-eo-docs.pages.dev Built from commit |
site/docs/demos/uplc-jvm.md (new "Demos" section in the site nav): - The idea: a compiler is a polymorphic Iso — S flat UPLC, A typed expression, B JVM code, T class file — and `iso.modify(Lower.lower)` is the compiler; `cborScript andThen iso` composes on-chain CBOR in front; the law that matters is the commuting square against the CEK machine. - Back and forth: an inline, theme-aware SVG of the round trip, then each stage on real output — UPLC text, A (`iso.get`), B (`Lower.lower`), the generated bytecode (`javap`), and a closure discharged back to a term, α-equal to the CEK machine's result — plus the Fibonacci validator read back as A. - Benchmarks: JMH tables for validation (2.5–3.6× on context-decoding validators, ~20× on the recursion-heavy one, 2–14× less allocation), Fibonacci scaling (10× → 25× → 18×), and translation cost with the break-even number of validations (~10–130). Supporting changes: - demos/uplc-jvm-bench: a JMH project (outside the root aggregate and the library's nightly sweep) — ValidatorBench, FibonacciBench, TranslateBench; setup checks both evaluators accept each transaction before timing. CI compiles it and checks its formatting and scalafix. - Validators module gains the shared off-chain fixtures (Parties, Contexts.spending, Scenarios); ValidatorSpec uses them. - JvmProgram.show renders B as pseudo-code; Demo becomes the stage-by-stage walkthrough the page quotes (timings moved to JMH). - README, CLAUDE.md, benchmarks.md cross-links; ci.yml regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
Scalus already has types for most of what the demo redefined: - SourceProgram works on Scalus' own decoded Term tree, numbered by object identity, instead of a mirror Node enum. Constants are looked up by node id. - A static type is Option[DefaultUni]. Builtin signatures come from Scalus' TypeScheme (numTypeVars / arity), replacing Ty, Step, Signature and the hand-written PrimOp table. The typed fast paths are the builtins Rt defines a method for. - Evaluation failures are Scalus' MachineError (UplcFailure is gone), and Exec.run returns scalus.uplc.eval.Result (Exec.Outcome is gone). Emit now uses Cojen Maker instead of raw ASM. Code is written against variables and method names, and Cojen resolves overloads, allocates locals and computes stack maps. That removes descriptors, the reflective descriptor table, the B-level case slots and Rt.spread. The emitter shrinks from 424 to 228 lines, and the generated bytecode is the same shape. The embedded program is now Base64. Cojen's constant-pool writer fails on the non-ASCII strings the Latin-1 encoding produced. asm-util's verifier is dropped too: the JVM verifies every class when Exec.load defines it. Main sources go from 2047 to 1766 lines, and the build depends on cojen-maker instead of asm + asm-util. The suite is unchanged: 1052 tests, 996 passed, 56 skipped. Re-run JMH: validation speed is the same within noise (2.6-3.4x on context-decoding validators, ~20x on Fibonacci). Translation is ~3x slower than with ASM (160-585 us), so break-even moves to ~25-260 validations. The docs page, README and CLAUDE.md are updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
Contributor
Benchmark A/BAllocation (B/op) — authoritative
442 more benchmarks
Timing (ns/op) — directional only, same-VM but shared runner
base_sha: Informational only · head |
Translation speed. The Cojen Maker version translated 2.3–3.9× slower than the original raw-ASM emitter. B is now ASM's own ClassNode, written directly by Lower with asm-commons' InstructionAdapter (no IR of our own, no library above ASM); reverseGet is ClassWriter. Strings ASM hashes are built once, switches use the last arm as default, Rt descriptors are precomputed. Translation is back to the raw-ASM emitter's cost (within noise; multisig ~20% slower), translate + load is level with it. Code size (code lines, no comments/blanks, vs the raw-ASM version): main 1425 → 991 (−30.5%), tests 534 → 330, validators 192 → 153, total 2218 → 1536 (−30.7%). JvmProgram, Emit and BuiltinCalls are gone. More Scalus reuse: - saturated builtins call scalus.uplc.builtin.Builtins directly on unboxed values; selectors, builtins used as values and everything else run Scalus' CEK meanings (BuiltinRuntime), with non-constant arguments swapped back by reference — no reflective twin; - chooseIndex runs the selector's own meaning on the branch indices; - builtin result types instantiate Scalus' TypeSchemes at the argument types, so `fstPair (unConstrData d)` is an integer and the tag match on it is a JVM switch. Validation speed (JMH, 3 forks): 8–21× the CEK machine on the context-decoding validators, 63× on Fibonacci; 2.9–7.1× faster than the raw-ASM version. Mechanisms: lazy Data lists (DataList / ConstrPair), direct Builtins calls, self-recursion as direct invokestatic, pure builtin lets computed where read, exact-arity applyAll, typed literals. Tests: ValidatorSpec is a loop over Scenarios.all (accepting and rejecting rows shared with the walkthrough and the benchmarks); CompilerSpec cases are one-liners; a vacuous `Rt.thunk` assertion now checks `Rt.closure`. dropList on a non-list now fails, as on the CEK machine. README, docs page (B as bytecode, new benchmark tables) and CLAUDE.md updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
- Rt.fail throws a MachineError without a stack trace. Filling it cost ~1.3 µs — more than a whole validation — so rejecting transactions took 1.8–2.7 µs against 0.4–1.4 µs for accepting ones; now 0.5–1.6 µs. - Exec.run's one catch is a Try turned into Scalus' Result; a compiler bug (AssertionError) still escapes, and discharge runs outside it. - Measured alternative, not taken: a result value from every call needs a check after each one — validation unchanged, translation 20–35% slower. Rt's scaladoc and the docs page say so. - Expr renders constants with Scalus' own PrettyPrinter.prettyValue; builtin names use scalus.utils.Utils.lowerFirst. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
The single Iso[Array[Byte], Array[Byte], TypedProgram, ClassNode] was
two unrelated functions packaged as an optic: its S was flat bytes, its
T class-file bytes, and nothing walked back. `cborScript andThen iso`
made that visible: cborScript's reverseGet was the identity on class
files, so the composed "iso" never returned to CBOR.
Now every stage keeps the script it came from — TypedProgram its
source, the class the script itself, in its constants — and each arrow
is a lawful monomorphic iso:
CborScript ≅ TypedProgram ≅ ClassNode ≅ JvmClassScript
typed lowering writing
UplcJvm.typed, bytecode (= typed andThen lowering) and classFile
(= bytecode andThen writing) all start at the script; classFile.get is
the compiler and classFile.reverseGet returns the script exactly.
- CborScript and JvmClassScript are opaque types over Scalus'
ByteString (value equality, so the laws compare with ==).
CborScript.of(program) reuses Scalus' cborEncoded / fromCbor;
CborScript.parse replaces UplcJvm.compile as the checked entry.
- OpticSpec runs cats-eo-laws' IsoTests on typed and classFile;
Oracle.compare checks the round trips on every program the suites
run, all 943 conformance programs included.
- writing (ClassWriter / ClassReader) is a byte-exact fixpoint because
generated code no longer has unreachable instructions: a function
that can only fail is never applied, and a call whose argument can
only fail after nothing but values is that failure. (Dead code left
constant-pool entries behind and reordered the pool on a rewrite.)
- TypedProgram is equal by script and body; its constructor is private.
- Thunk folds into Lam (arity 0); chooseList and trace run their CEK
meanings; one duplicated CompilerSpec case removed.
Code lines vs the raw-ASM version: main 1425 → 997 (−30.0%), total
2218 → 1549 (−30.2%). Translation unchanged within noise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
A doc comment ran one character past maxColumn; the local check had passed from scalafmt's incremental cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
The middle of the chain is now `JvmClass`, an opaque type over ASM's ClassNode, so the optics read in the demo's own vocabulary: CborScript ≅ TypedProgram ≅ JvmClass ≅ JvmClassScript Only `Lower.lower` and `writing.reverseGet` make one, so every JvmClass embeds its script; `.tree` exposes ASM's tree for inspection. Docs, README, CLAUDE.md and the build comment follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
…Bytecode Per review: - typed -> toTyped (CborScript ≅ TypedProgram) - writing -> compiling (JvmClass ≅ JvmClassScript) - bytecode -> toJvmClass (CborScript ≅ JvmClass) - classFile -> toBytecode (CborScript ≅ JvmClassScript) `compiling`'s two halves are now public as `UplcJvm.compile` and `UplcJvm.decompile`; the Oracle's class-file fixpoint check uses them directly. Docs, README, the walkthrough's labels and CLAUDE.md follow, and the last stage-letter labels ((A), (B), (T)) are gone from the comments and README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
Per review: - No unsafeFromArray (ByteString or IArray) or unsafeWrapArray left in the demo's main code. Scalus' unsafeFromArray skips a defensive copy; the scripts now pay that copy. - CborScript.of returns Either: Scalus' flat encoder throws on a program with a free variable, and the conformance suite used to wrap the call in a Try. Tests and benches unwrap it through one helper each. - Lower.direct no longer calls Option.get: an Option for-comprehension with cats' traverse replaces it. - chunks/fromChunks are now CborScript.asConstants/fromConstants, with the reason in the doc: a class carries its script, a class file has no byte-array constants, and one string constant holds at most 65535 bytes. - SourceProgram.decode is a right fold over a shared `subterms` instead of writes into arrays. Measured, the fold is 2-3x slower to decode and translation 2-9% slower; validation never decodes. children(n) is one unfold over subtree sizes. FreeVars walks `subterms` too but keeps its mutable set: immutable sets were 2.5x slower. - Docs: the compiler does no optimisation (no constant folding, inlining, dead-code elimination, CSE, specialisation or tail-call elimination); translation table re-measured. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
- demoUplcJvm now compiles with `-opt -opt-inline:**,!scala.**`. It is unpublished, so it may inline from Scalus, ASM, cats and eo. Inlining from the Scala library is left out: measured with the demo's JMH benches, it made the short validations 12-27% slower (PrintInlining shows the inlined library code makes Rt.force too big for the JIT to inline). Everything else measures neutral to ~5% faster. The generated classes are byte-identical either way. - Published modules (scala3LibrarySettings) add `-opt-inline:dev.constructive.eo.**`: the inliner may copy code from eo's own packages only, never from cats, the JDK or the Scala library. This replaces core's `<sources>` scope and extends inlining to every library module. Full test suite green on JDK 25. - Docs: the demo's "no optimisation" note is about UPLC programs; the benchmark tables are re-measured with the new build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
…ealed - compiling and the composed toBytecode are prisms: a class can exceed the JVM's 64 KB method or 65 535-constant class limits, and then there is no class file. UplcJvm.compile returns the limit as a Left (the one place ASM's MethodTooLargeException / ClassTooLargeException become a value); toBytecode.getOption is the compiler. - CborScript, JvmClassScript and JvmClass hand out copies only (bytes, tree), so a value cannot change after the compiler made or checked it and reverseGet always walks back. - Compiling cannot overflow the JVM stack: decoding, typing and lowering are walks of Steps (Step.scala), plain calls while shallow and cats.Eval on the heap from 128 calls deep; every recursive call goes through Walk.nested. Trampolining every node instead doubled translation time; this costs 13-19% of it. A closure's captures are a fold over its subtree's ids, so FreeVars is gone. Programs nested 100 000 deep compile on a 1 MB stack. - OpticSpec: PrismTests for toBytecode (a too-large program among the scripts), the miss, the copies, and the 100 000-deep programs. - Docs: README, site page and CLAUDE.md describe two isos and a prism, sealing, stack safety and Scalus' still-recursive flat codec; the translation table is re-measured. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
…ns miss Scalus decodes any declared Plutus Core version, and the compiler used to type every script as if it were 1.1.0: a script declaring 2.0.0 or 7.3.9 parsed, compiled and ran. The ledger only takes the versions Plutus V3 has at protocol version 11, 1.0.0 and 1.1.0 (Semantics.versions). - toTyped is a prism: getOption misses on a script of any other version; reverseGet stays the script the program was read from. toJvmClass and toBytecode compose to prisms; lowering stays an iso, typing the script a class embeds without the check (only a typed program is lowered). - SourceProgram keeps the decoded program's version. - OpticSpec runs PrismTests on toTyped, with a 2.0.0 script among the inputs, and checks the miss. Tests reach the stages through Oracle.typed, jvmClass and classFile. - Docs: the chain is CborScript ⇀ TypedProgram ≅ JvmClass ⇀ JvmClassScript. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
CborScript ⇀ TypedProgram ≅ JvmClass ⇀ JvmClassScript: toTyped and compiling are prisms since the last two commits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A demo compiler from Untyped Plutus Core (UPLC, the language Cardano smart contracts run in) to JVM class files. It is built from lawful eo optics, all starting at the script as the ledger stores it: a prism, an iso and a prism.
Why each arrow walks back. Every stage keeps the script it came from. A
TypedProgramkeeps its source. A class stores the script as Base64 string constants: a class file has no byte-array constants, and one string constant holds at most 65535 bytes. SoreverseGetwalks back exactly.Two prisms around an iso.
loweringis an iso: there and back is the identity in both directions, on everything the compiler produces. The other two are prisms, total on the way back:toTyped: the bytes can declare any Plutus Core version, and Scalus decodes them all. The ledger only takes the versions Plutus V3 has at protocol version 11, 1.0.0 and 1.1.0 (Semantics.versions), so a script of any other version has no typed program. Before this, a script declaring 2.0.0 or 7.3.9 parsed, compiled and ran as if it were 1.1.0.compiling: the JVM caps a method at 64 KB of bytecode and a class at 65 535 constants, so a class can be too large for a class file, and then there is none.Prism, iso and prism compose to a prism, so
toJvmClassand the compiler,toBytecode, are prisms too.Sealed values.
CborScriptandJvmClassScriptare opaque types over Scalus'ByteString. The two byte encodings can't be swapped by accident, and the laws can compare them with==.JvmClassis an opaque type over ASM'sClassNode. Only the compiler andUplcJvm.decompilemake one, so it always embeds its script.bytesand.treeare copies, andCborScript.parsechecks a copy of the caller's array. A value cannot change after the compiler made or checked it, so the way back always works.CborScript.ofreturnsLeftfor a program Scalus can't flat-encode, such as one with a free variable.CborScript.parsereturnsLeftfor bytes that don't decode.unsafe*constructors.Stack safety. Compiling cannot overflow the JVM stack, however deep the program. Decoding, typing and lowering are walks of
Steps (Step.scala): plain calls while a walk is shallow, and from 128 calls deep the rest runs on the heap, fromcats.Eval's loop. Every recursive call goes throughWalk.nested. A program nested 100 000 deep compiles on a thread's default 1 MB stack.Evalmade translation 2–2.7× slower. Trampolining only when deep costs 13–19%.FreeVarsis gone.Typer (
toTyped). Works directly on Scalus' decodedTermtree. It recognises what UPLC compilers emit for constructs UPLC lacks:let, andifover delayed branches;let-bound functions and forced builtins;case (constr k args…)multi-argument call.Every rewrite is exact under the CEK machine's evaluation order. An expression that can only fail is never followed by code, so the generated class has no unreachable instructions.
Hybrid typing. A static type is a Scalus
DefaultUni. A builtin's signature is Scalus' ownTypeScheme, instantiated at its argument types, sofstPair (unConstrData d)is anintegerand the constructor-tag match on it becomes a JVM switch. Integers, booleans, bytestrings and strings stay unboxed wherever the typer can prove them.Lowering (
lowering). Writes bytecode straight into ASM's class tree (theJvmClass) with asm-commons'InstructionAdapter. It converts closures using a capture layout fixed by the source, so results can be "discharged" back to terms and compared with CEK results. A saturated builtin is a direct call of thescalus.uplc.builtin.Builtinsfunction Scalus'CardanoBuiltinswraps.Compiling (
compiling).UplcJvm.compileis ASM'sClassWriter, which computes the frames, and returnsLeftfor a class over a JVM limit (the one place ASM'sMethodTooLargeException/ClassTooLargeExceptionbecome a value).UplcJvm.decompileis itsClassReader. Both are public. A class file survives being read and written again byte for byte.Runtime. List and pair builtins read
Datalazily. Builtins used as values, and the rest, run Scalus' CEK meanings. A failure is a stacklessMachineError, turned into Scalus'ResultbyExec.run.No optimisation of UPLC programs. The compiler translates; it does not optimise. There is no constant folding, inlining, dead-code elimination, CSE, specialisation or tail-call elimination: every operation runs, in the CEK machine's order. The speed-ups come from running as JVM bytecode under the JIT, with values unboxed and calls made directly where the types allow.
Correctness
Semantics.scala: Plutus V3, PV11, variant E, the configuration Scalus 1.3.0 runs the conformance suite under, and the Plutus Core versions it accepts (1.0.0, 1.1.0).sbt demoUplcJvm/testruns 1067 tests: 1011 passed, 56 skipped, 0 failed.PrismTestschecktoTypedandtoBytecode, over every validator, a closure-valued program, a script declaring Plutus Core 2.0.0, and a program too large for a class file. (JvmClasshas no equality, soloweringis checked by the round trips rather thanIsoTests.)Oracle.comparealso checks the round trips on every program it runs.OpticSpecchecks that changing the arrays and trees a value hands out changes nothing, and compiles programs nested 100 000 deep (force (delay …), builtin calls, lambdas) on a 1 MB stack: each gives a class file, or the JVM limit its class exceeds — never aStackOverflowError.1.63.0.0, vendored. All 943 evaluation cases agree with the CEK machine. The 56 skips are parser-only tests and one bare free-variable program.scalus-plugin. There are 20 accepting and rejecting V3 transactions, each checked compiled-then-applied and with the context baked in.Benchmarks (JMH,
demos/uplc-jvm-bench, 3 forks)translate). That is about 15% more than the first raw-ASM emitter cost, and about 40% of the intermediate Cojen Maker version's cost.Termtree. It is 2–3× slower than writing into arrays, at a cost of a few microseconds per script.Size
Code lines (no comments or blanks) against the first raw-ASM version:
The prisms, sealing and stack safety added 115 lines of main code (the 37-line
Step.scala, plusLowerandTyperwritten as walks of steps, the prisms and the copies) and 80 lines of tests (the prism laws, misses, copies and depth tests).Build and CI
demoUplcJvmdepends on Scalus andasm-commons. Its tests useasm-utiland cats-eo-laws.demoUplcJvmValidatorsholds the plugin-compiled validators and shared fixtures.demoUplcJvmBenchis the JMH project.ci.ymlruns the demo tests and compiles the benchmarks on each JDK lane.demoUplcJvmcompiles with-opt -opt-inline:**,!scala.**. It is unpublished, so it may inline from Scalus, ASM, cats and eo. Inlining from the Scala library is excluded because it made the short validations 12–27% slower: the inlined code makesRt.forcetoo big for the JIT to inline (seen with-XX:+PrintInlining). Every other scope measured neutral to ~5% faster. The generated classes are byte-identical either way.scala3LibrarySettingsadds-opt-inline:dev.constructive.eo.**, so every library module may inline code from eo's own packages, and never from cats, the JDK or the Scala library. This replacescore's former<sources>scope.Limitations
letboundaries only. A class that still exceeds a JVM limit has no class file:toBytecode.getOptionisNone.Execuses a thread with a 512 MB stack.🤖 Generated with Claude Code
https://claude.ai/code/session_01JG5LJsPA524HKbSMvjStrv