diff --git a/docs/ModeMath-architecture-study.md b/docs/ModeMath-architecture-study.md new file mode 100644 index 00000000..b573cb55 --- /dev/null +++ b/docs/ModeMath-architecture-study.md @@ -0,0 +1,69 @@ +# ModeMath architecture study (issue #273) + +## Decision + +ModeMath is a useful source of focused techniques, not a replacement architecture for CSharpMath. We should adapt selected ideas behind existing compatibility and editor invariants, and land them incrementally behind the relevant work items. + +## Reproducibility + +The comparison used CSharpMath revision `e1cb9d62` (full commit `e1cb9d6270289ec1903022ba7a3275ba52a3d819`, origin/master) and ModeMath revision `1ba976e0125fa03f41d6fe4f5aa4c60df89cfb10`, on Windows `10.0.19045`, `win-x64`, .NET SDK `10.0.302` and runtime `10.0.10`. The ModeMath library build was clean. The capture workflow requires both source checkouts to be at these commits; it permits only the untracked study artifacts in the CSharpMath worktree and rejects tracked product/harness changes. The solution test-project restore has an `NU1605` dependency caveat; with `-p:NoWarn=NU1605`, ModeMath tests were `630/630` and CSharpMath Core tests were `1515/1515`. + +Representative commands (use local variables for paths): + +```powershell +$ModeMathRoot = 'C:\path\to\ModeMath-1ba976e' +$ModeMathPackages = 'C:\path\to\ModeMath-packages' +dotnet build "$ModeMathRoot\ModeMath\ModeMath.fsproj" -c Release +dotnet test "$ModeMathRoot\ModeMath.slnx" -c Release -p:NoWarn=NU1605 +dotnet test CSharpMath.Core.Tests\CSharpMath.Core.Tests.csproj -c Release -p:NoWarn=NU1605 +dotnet restore research\ModeMathStudyBench\ModeMathStudyBench.fsproj --source $ModeMathPackages --source https://api.nuget.org/v3/index.json -p:NoWarn=NU1605 +dotnet build research\ModeMathStudyBench\ModeMathStudyBench.fsproj -c Release --no-restore -p:GeneratePackageOnBuild=false +``` + +The study-only package is `ModeMath.0.12.0-csharpmath-study.1ba976e.nupkg`; verify its SHA-256 as `0B03977732F2CE13EC9A290DD28A30DAFEF48E81ED36FC0B38670B4342700926` before restore. The capture script enforces the exact filename, hash, package version, clean source checkout, and pinned source revision, then restores/builds the Release harness from that local package before `--no-build` capture. The unique prerelease version prevents a silent fallback to public `0.12.0`. + +## Feature matrix and decisions + +| Area | ModeMath observation | CSharpMath decision | +|---|---|---| +| Parsing | Small explicit parser; rejects unknown input | Study/adapt where it preserves public behavior | +| LaTeX roundtrip | `Read`/`Write`, canonical braces | Prototype against compatibility corpus | +| Layout/rendering | `Layout` then `Painter`, generated metrics | Adapt techniques; retain backend contracts | +| Tentative brackets | Editor-state tentative delimiters | Adapt incrementally under #82 | +| Navigation | Cursor over immutable MA tree | Reject wholesale; adapt internal invariants/transactions | +| Decorated editing | Limited editor model | Keep separate prototype; caret/empty-slot work under #158 | +| Undo/redo | Transaction-oriented editor state | Adapt transaction ideas after invariant tests | +| Typed functions/radicals | Reversible, longest-token recognition | Prototype under #83; reject substring/opaque semantics | +| Public compatibility | New API and narrower accepted input | Preserve CSharpMath public API and accepted behavior | + +Generated `MathFontData` is the prototype hybrid default path under #191/#293; preserve the runtime `LocalTypeface` fallback. This keeps generated data useful without making embedded data the only deployment path. + +## Complexity and size + +The reproducible source-only count is 4,914 ModeMath runtime lines: 4,133 handwritten lines in the ten `ModeMath/*.fs` files other than `MathFontData.g.fs`, plus 781 generated lines in `MathFontData.g.fs`. Its `tools/MathTableGen/*.fs` generator is 658 lines and `tools/trimfont.py` is 104 lines. The count is physical lines from tracked files, including blank/comment lines. Exact PowerShell counting form (run from each repository root) is: + +```powershell +$paths = git ls-files 'ModeMath/*.fs' # use the corresponding glob for each tree +$lines = 0; foreach ($path in $paths) { $lines += (git show "HEAD:$path" | Measure-Object).Count } +"$($paths.Count) files, $lines lines" +``` + +It excludes project files, tests, examples, build output, and all other directories. For context, the same command with `CSharpMath/**/*.cs`/`CSharpMath/*.cs` gives CSharpMath 84 tracked files and 7,862 lines; with `CSharpMath.Rendering/**/*.cs`/`CSharpMath.Rendering/*.cs` it gives Rendering 27 files and 2,040 lines (tests and build output excluded). File/line counts are not an engineering-effort or quality measure. + +## Font and package evidence + +Raw embedded faces measured 87,324 B for ModeMath versus 786,744 B for CSharpMath: 699,420 B (88.9%) less for ModeMath. This is not package-size or runtime-memory proof and excludes dependency/feature differences. ModeMath package was 251,207 B and `ModeMath.dll` 458,752 B. CSharpMath's compressed chain was `117,571 + 834,847 + 22,512 ~= 974,930 B`; its DLLs were `242,688 + 1,798,144 + 11,264 B`. These are not apples-to-apples: dependencies, platforms, APIs, and supported features differ. + +## Benchmark evidence + +The normalized three-formula harness aggregate is checked in at [results-captured.md](../research/ModeMathStudyBench/results-captured.md), computed from 10 fresh cold samples and 3 steady samples per library. Cold medians were ModeMath 229.681 ms (179.948-401.301) versus CSharpMath 513.767 ms (367.005-737.039); managed allocation 267,800 B versus 22,053,456 B; retained 90,192 B versus 3,896,944 B. Steady per-formula parse was 10.117 versus 27.631 us and 2.833 versus 5.703 KB allocation; layout 18.030 versus 69.908 us and 6.682 versus 31.749 KB; draw 35.735 versus 182.400 us and 1.586 versus 10.094 KB. A three-key editor sequence was 22.666 versus 39.297 us and 6.758 versus 25.307 KB. Per-metric ranges are in the checked-in aggregate. Results remain directional/manual, managed-only, use different editor histories/timers, and are not a performance gate. + +These numbers are directional/manual, managed-only, use different editor histories and timers, and omit Skia native allocation, JIT/code pages, process startup, and platform variance. They provide no performance gate. + +## Licensing inventory + +ModeMath source is MIT (Summatic). Its Latin Modern-derived face is under the GUST Font License/LPPL-derived obligations, including derived-font renaming; AMS faces carry OFL notices and reserved-name restrictions. Generated metrics and outlines retain font provenance. Before shipping any adapted/generated asset, CSharpMath needs a third-party-notices and package audit. This inventory is not legal advice. + +## Unknowns and next experiments + +Unknowns include broad LaTeX compatibility, accessibility/selection semantics, malformed-input recovery, font fallback coverage, trimming/AOT behavior, native memory, and cross-platform rendering parity. Next experiments should: (1) differential-test parser and roundtrip behavior against the existing corpus; (2) prototype #82/#83/#158/#191/#293 behind focused tests; (3) measure AOT/trimming and native memory on supported platforms; (4) audit licenses and generated-file provenance; and (5) compare editor histories with a shared scripted workload. None should become a gate until methodology and feature parity are agreed. diff --git a/research/ModeMathStudyBench/ModeMathStudyBench.fsproj b/research/ModeMathStudyBench/ModeMathStudyBench.fsproj new file mode 100644 index 00000000..b2d9b822 --- /dev/null +++ b/research/ModeMathStudyBench/ModeMathStudyBench.fsproj @@ -0,0 +1,14 @@ + + + Exe + net10.0 + enable + + + + + + + + + diff --git a/research/ModeMathStudyBench/Program.fs b/research/ModeMathStudyBench/Program.fs new file mode 100644 index 00000000..d8d81be5 --- /dev/null +++ b/research/ModeMathStudyBench/Program.fs @@ -0,0 +1,137 @@ +open System +open System.Diagnostics +open SkiaSharp +open ModeMath + +type Metric = { + Name: string + Iterations: int + ElapsedNanoseconds: int64 + AllocatedBytes: int64 + RetainedBytes: int64 + Gen0: int + Gen1: int + Gen2: int +} + +let corpus = + [| @"x^2 + y_1" + @"\frac{-b \pm \sqrt{b^2 - 4ac}}{2a}" + @"\int_0^1 x^2\,dx" |] + +let collect () = + GC.Collect() + GC.WaitForPendingFinalizers() + GC.Collect() + +let measure name iterations action = + collect () + let retainedBefore = GC.GetTotalMemory(true) + let allocatedBefore = GC.GetTotalAllocatedBytes(true) + let gen0 = GC.CollectionCount(0) + let gen1 = GC.CollectionCount(1) + let gen2 = GC.CollectionCount(2) + let stopwatch = Stopwatch.StartNew() + for _ = 1 to iterations do action () + stopwatch.Stop() + let allocated = GC.GetTotalAllocatedBytes(true) - allocatedBefore + collect () + { + Name = name + Iterations = iterations + ElapsedNanoseconds = stopwatch.ElapsedTicks * 1_000_000_000L / Stopwatch.Frequency + AllocatedBytes = allocated + RetainedBytes = GC.GetTotalMemory(true) - retainedBefore + Gen0 = GC.CollectionCount(0) - gen0 + Gen1 = GC.CollectionCount(1) - gen1 + Gen2 = GC.CollectionCount(2) - gen2 + } + +let print metric = + printfn + "{\"name\":\"%s\",\"iterations\":%d,\"elapsedNanoseconds\":%d,\"allocatedBytes\":%d,\"retainedBytes\":%d,\"gen0\":%d,\"gen1\":%d,\"gen2\":%d}" + metric.Name metric.Iterations metric.ElapsedNanoseconds metric.AllocatedBytes + metric.RetainedBytes metric.Gen0 metric.Gen1 metric.Gen2 + +let modeRead latex = + match Latex.Read latex with + | Ok formula -> formula + | Error error -> failwith error.Message + +let csharpRead latex = + CSharpMath.Atom.LaTeXParser.MathListFromLaTeX(latex).Match( + (fun formula -> formula), + (fun error -> failwith error)) + +let coldModeMath () = + measure "modemath-cold-render" 1 (fun () -> + let formula = modeRead corpus.[1] + let placed = (Layout 32f).Of formula + use painter = Painter.Embedded() + use surface = SKSurface.Create(SKImageInfo(800, 240)) + use paint = new SKPaint() + painter.Draw(placed, surface.Canvas, 0f, placed.Ascent, paint) + surface.Canvas.Flush()) + |> print + +let coldCSharpMath () = + measure "csharpmath-cold-render" 1 (fun () -> + let painter = CSharpMath.SkiaSharp.MathPainter(FontSize = 24f, LaTeX = corpus.[1]) + use surface = SKSurface.Create(SKImageInfo(800, 240)) + painter.Draw(surface.Canvas, CSharpMath.Rendering.FrontEnd.TextAlignment.TopLeft) + surface.Canvas.Flush()) + |> print + +let steadyModeMath () = + let parsed = corpus |> Array.map modeRead + let layout = Layout 32f + let placed = parsed |> Array.map layout.Of + use painter = Painter.Embedded() + use surface = SKSurface.Create(SKImageInfo(800, 240)) + use paint = new SKPaint() + measure "modemath-parse-batch" 10_000 (fun () -> corpus |> Array.iter (modeRead >> ignore)) |> print + measure "modemath-layout-batch" 5_000 (fun () -> parsed |> Array.iter (layout.Of >> ignore)) |> print + measure "modemath-draw-batch" 2_000 (fun () -> + placed |> Array.iter (fun formula -> painter.Draw(formula, surface.Canvas, 0f, formula.Ascent, paint))) + |> print + measure "modemath-editor-x2" 2_000 (fun () -> + let editor = Editor(layout, MA.Empty) + editor.Press(MathKey.Character 'x') |> ignore + editor.Press(MathKey.Superscript) |> ignore + editor.Press(MathKey.Character '2') |> ignore) + |> print + +let steadyCSharpMath () = + let parsed = corpus |> Array.map csharpRead + let painter = CSharpMath.SkiaSharp.MathPainter(FontSize = 24f) + parsed |> Array.iter (fun formula -> painter.Content <- formula; painter.Measure() |> ignore) + let painters = + parsed + |> Array.map (fun formula -> + let prepared = CSharpMath.SkiaSharp.MathPainter(FontSize = 24f, Content = formula) + prepared.Measure() |> ignore + prepared) + use surface = SKSurface.Create(SKImageInfo(800, 240)) + let fonts = CSharpMath.Rendering.BackEnd.Fonts(Array.empty, 24f) + measure "csharpmath-parse-batch" 10_000 (fun () -> corpus |> Array.iter (csharpRead >> ignore)) |> print + measure "csharpmath-layout-batch" 5_000 (fun () -> + parsed |> Array.iter (fun formula -> painter.Content <- formula; painter.Measure() |> ignore)) + |> print + measure "csharpmath-draw-batch" 2_000 (fun () -> + painters |> Array.iter (fun prepared -> prepared.Draw(surface.Canvas, 0f, 120f))) + |> print + measure "csharpmath-editor-x2" 2_000 (fun () -> + use keyboard = + new CSharpMath.Editor.MathKeyboard( + CSharpMath.Rendering.BackEnd.TypesettingContext.Instance, fonts, 1_000_000_000.) + keyboard.KeyPress(CSharpMath.Editor.MathKeyboardInput.SmallX) + keyboard.KeyPress(CSharpMath.Editor.MathKeyboardInput.Power) + keyboard.KeyPress(CSharpMath.Editor.MathKeyboardInput.D2)) + |> print + +match Environment.GetCommandLineArgs() |> Array.tryLast with +| Some "cold-modemath" -> coldModeMath () +| Some "cold-csharpmath" -> coldCSharpMath () +| Some "steady-modemath" -> steadyModeMath () +| Some "steady-csharpmath" -> steadyCSharpMath () +| _ -> failwith "Expected cold-modemath, cold-csharpmath, steady-modemath, or steady-csharpmath" diff --git a/research/ModeMathStudyBench/README.md b/research/ModeMathStudyBench/README.md new file mode 100644 index 00000000..01cfdcaf --- /dev/null +++ b/research/ModeMathStudyBench/README.md @@ -0,0 +1,47 @@ +# ModeMath comparison harness + +This research-only harness supports the architecture study in issue #273. It is deliberately not a performance gate: it reports raw wall time and managed allocation/retention for the same three-formula corpus on both libraries. The checked-in aggregate in `results-captured.md` is a captured result, not a rerun or a statistically complete sample archive. + +Set these local-only variables to your checkout locations (do not commit machine-specific paths): + +```powershell +$ModeMathRoot = 'C:\path\to\ModeMath-1ba976e' +$ModeMathPackages = 'C:\path\to\ModeMath-packages' +$ModeMathVersion = '0.12.0-csharpmath-study.1ba976e' +git clone https://github.com/SummaticLtd/ModeMath.git $ModeMathRoot +git -C $ModeMathRoot checkout --detach 1ba976e0125fa03f41d6fe4f5aa4c60df89cfb10 +dotnet build "$ModeMathRoot\ModeMath\ModeMath.fsproj" -c Release +dotnet pack "$ModeMathRoot\ModeMath\ModeMath.fsproj" -c Release --no-build -p:PackageVersion=$ModeMathVersion -o $ModeMathPackages +Get-FileHash "$ModeMathPackages\ModeMath.$ModeMathVersion.nupkg" -Algorithm SHA256 +``` + +Restore and build this harness from the CSharpMath repository. The local package source must come first so the pinned package is selected; keep nuget.org for transitive dependencies: + +```powershell +dotnet restore research\ModeMathStudyBench\ModeMathStudyBench.fsproj ` + --source $ModeMathPackages ` + --source "https://api.nuget.org/v3/index.json" ` + --no-cache --force-evaluate -p:NoWarn=NU1605 +dotnet build research\ModeMathStudyBench\ModeMathStudyBench.fsproj ` + -c Release --no-restore -p:GeneratePackageOnBuild=false +``` + +`NU1605` is suppressed only for this pinned study: the current `SimpleTests` dependency requests a newer `FSharp.Core` than ModeMath's pinned floor. The ModeMath library itself builds without that suppression. + +Each cold sample must be a fresh process. The checked-in `capture.ps1` validates both pinned revisions and the clean tracked product/harness source state, then restores into a fresh isolated package cache from the exact local package and builds the Release harness before capture. It alternates ten cold samples per library and captures three steady samples per library. It preserves every JSON metric line in NDJSON with sample, library, mode, revisions, environment, and package hash metadata. Run it: + +```powershell +powershell -ExecutionPolicy Bypass -File research\ModeMathStudyBench\capture.ps1 ` + -ModeMathRoot $ModeMathRoot -PackageDirectory $ModeMathPackages +``` + +Run steady-state parse, layout, draw, and three-key editor updates separately: + +```powershell +dotnet run --project research\ModeMathStudyBench -c Release --no-build -- steady-modemath +dotnet run --project research\ModeMathStudyBench -c Release --no-build -- steady-csharpmath +``` + +The harness intentionally measures managed allocations only. Skia native allocations, JIT/code pages, process startup, and platform variance are outside these numbers and must be stated alongside any result. + +Each command writes one JSON object per metric to stdout. `capture.ps1` saves stdout as `raw-results.ndjson`; the aggregate table in `results-captured.md` is computed from that file. `RetainedBytes` is a post-GC process measurement and is especially sensitive to runtime/library initialization. The ModeMath 32 px em matches CSharpMath's 24 pt at 96 DPI (approximately 32 px); keep this equivalence when comparing runs. diff --git a/research/ModeMathStudyBench/aggregate.ps1 b/research/ModeMathStudyBench/aggregate.ps1 new file mode 100644 index 00000000..a4018daa --- /dev/null +++ b/research/ModeMathStudyBench/aggregate.ps1 @@ -0,0 +1,58 @@ +param( + [string]$InputPath = "$PSScriptRoot\raw-results.ndjson", + [string]$Output = "$PSScriptRoot\aggregates.json", + [switch]$Check +) +$ErrorActionPreference = 'Stop' +$rows = Get-Content $InputPath | ForEach-Object { $_ | ConvertFrom-Json } | Where-Object record -eq 'metric' +$expectedLibraries = @('modemath', 'csharpmath') +$expectedSteadySuffixes = @('parse-batch', 'layout-batch', 'draw-batch', 'editor-x2') +if ($rows.Count -ne 44) { throw "Expected exactly 44 canonical metric rows; found $($rows.Count)" } +foreach ($library in $expectedLibraries) { + $cold = @($rows | Where-Object { $_.library -eq $library -and $_.mode -eq "cold-$library" }) + $coldName = "$library-cold-render" + $coldSamples = @($cold | ForEach-Object { [int]$_.sample } | Sort-Object) + if ($cold.Count -ne 10 -or (($cold | Select-Object -ExpandProperty name -Unique) -join ',') -ne $coldName -or (($coldSamples -join ',') -ne ((1..10) -join ','))) { + throw "Expected exactly 10 rows for $library cold capture" + } + $steady = @($rows | Where-Object { $_.library -eq $library -and $_.mode -eq "steady-$library" }) + $steadyNames = @($steady | Select-Object -ExpandProperty name -Unique) + $expectedNames = @($expectedSteadySuffixes | ForEach-Object { "$library-$_" } | Sort-Object) + if ($steady.Count -ne 12 -or (($steadyNames | Sort-Object) -join ',') -ne ($expectedNames -join ',')) { + throw "Expected four distinct steady metrics, three rows each, for $library" + } + foreach ($name in $expectedNames) { + $metric = @($steady | Where-Object name -eq $name) + $samples = @($metric | ForEach-Object { [int]$_.sample } | Sort-Object) + if ($metric.Count -ne 3 -or (($samples -join ',') -ne ((1..3) -join ','))) { throw "Expected samples 1..3 exactly once for $name" } + } +} +$unexpected = @($rows | Where-Object { $_.library -notin $expectedLibraries -or $_.mode -notin @("cold-$($_.library)", "steady-$($_.library)") }) +if ($unexpected.Count -ne 0) { throw 'Unexpected metric group identity in raw results' } +$formulaCount = 3 +$groups = $rows | Group-Object library, name +$result = foreach ($group in $groups) { + $sample = $group.Group + $isCold = $sample[0].name -like '*cold*' + $denominator = if ($isCold) { 1 } elseif ($sample[0].name -like '*editor*') { $sample[0].iterations } else { $sample[0].iterations * $formulaCount } + $times = @($sample | ForEach-Object { [double]$_.elapsedNanoseconds / 1000.0 / $denominator }) | Sort-Object + $alloc = @($sample | ForEach-Object { [double]$_.allocatedBytes / $denominator }) | Sort-Object + $retained = @($sample | ForEach-Object { [double]$_.retainedBytes }) | Sort-Object + $middle = [math]::Floor($times.Count / 2) + $timeMedian = if ($times.Count % 2) { $times[$middle] } else { ($times[$middle - 1] + $times[$middle]) / 2 } + $middle = [math]::Floor($alloc.Count / 2) + $allocMedian = if ($alloc.Count % 2) { $alloc[$middle] } else { ($alloc[$middle - 1] + $alloc[$middle]) / 2 } + $middle = [math]::Floor($retained.Count / 2) + $retainedMedian = if ($retained.Count % 2) { $retained[$middle] } else { ($retained[$middle - 1] + $retained[$middle]) / 2 } + [ordered]@{ library = $sample[0].library; metric = $sample[0].name; samples = $sample.Count; elapsedUsMedian = $timeMedian; elapsedUsMin = $times[0]; elapsedUsMax = $times[-1]; allocatedBytesMedian = $allocMedian; allocatedBytesMin = $alloc[0]; allocatedBytesMax = $alloc[-1]; retainedBytesMedian = $retainedMedian; retainedBytesMin = $retained[0]; retainedBytesMax = $retained[-1] } +} +$json = @($result) | ConvertTo-Json -Depth 4 +if ($Check) { + if (-not (Test-Path $Output)) { throw "Missing aggregate artifact: $Output" } + $expected = @(Get-Content $Output -Raw | ConvertFrom-Json) + if (($json | ConvertFrom-Json | ConvertTo-Json -Depth 4) -ne ($expected | ConvertTo-Json -Depth 4)) { throw 'Aggregate artifact does not match raw-results.ndjson' } + Write-Host 'Aggregate check passed.' +} else { + Set-Content -Path $Output -Value $json -Encoding utf8 + Write-Output $json +} diff --git a/research/ModeMathStudyBench/aggregates.json b/research/ModeMathStudyBench/aggregates.json new file mode 100644 index 00000000..cb78201d --- /dev/null +++ b/research/ModeMathStudyBench/aggregates.json @@ -0,0 +1,142 @@ +[ + { + "library": "modemath", + "metric": "modemath-cold-render", + "samples": 10, + "elapsedUsMedian": 229680.8, + "elapsedUsMin": 179948.2, + "elapsedUsMax": 401301.2, + "allocatedBytesMedian": 267800, + "allocatedBytesMin": 264704, + "allocatedBytesMax": 270896, + "retainedBytesMedian": 90192, + "retainedBytesMin": 87120, + "retainedBytesMax": 98416 + }, + { + "library": "csharpmath", + "metric": "csharpmath-cold-render", + "samples": 10, + "elapsedUsMedian": 513767.3, + "elapsedUsMin": 367005.4, + "elapsedUsMax": 737039.1, + "allocatedBytesMedian": 22053456, + "allocatedBytesMin": 22053456, + "allocatedBytesMax": 22065792, + "retainedBytesMedian": 3896944, + "retainedBytesMin": 3890800, + "retainedBytesMax": 3905168 + }, + { + "library": "modemath", + "metric": "modemath-parse-batch", + "samples": 3, + "elapsedUsMedian": 10.117443333333332, + "elapsedUsMin": 7.3257766666666662, + "elapsedUsMax": 11.556143333333333, + "allocatedBytesMedian": 2901.3346666666666, + "allocatedBytesMin": 2901.3346666666666, + "allocatedBytesMax": 2901.3346666666666, + "retainedBytesMedian": 376, + "retainedBytesMin": 376, + "retainedBytesMax": 8600 + }, + { + "library": "modemath", + "metric": "modemath-layout-batch", + "samples": 3, + "elapsedUsMedian": 18.029760000000003, + "elapsedUsMin": 14.277619999999999, + "elapsedUsMax": 22.914360000000002, + "allocatedBytesMedian": 6842.6933333333336, + "allocatedBytesMin": 6842.6917333333331, + "allocatedBytesMax": 6843.0821333333333, + "retainedBytesMedian": 0, + "retainedBytesMin": -5152, + "retainedBytesMax": 24 + }, + { + "library": "modemath", + "metric": "modemath-draw-batch", + "samples": 3, + "elapsedUsMedian": 35.735383333333331, + "elapsedUsMin": 33.927633333333333, + "elapsedUsMax": 38.61825, + "allocatedBytesMedian": 1624.4573333333333, + "allocatedBytesMin": 1624.4013333333332, + "allocatedBytesMax": 1624.4573333333333, + "retainedBytesMedian": 1312, + "retainedBytesMin": -6912, + "retainedBytesMax": 1312 + }, + { + "library": "modemath", + "metric": "modemath-editor-x2", + "samples": 3, + "elapsedUsMedian": 22.6655, + "elapsedUsMin": 16.61645, + "elapsedUsMax": 49.191050000000004, + "allocatedBytesMedian": 6920.356, + "allocatedBytesMin": 6920.188, + "allocatedBytesMax": 6920.356, + "retainedBytesMedian": 336, + "retainedBytesMin": 336, + "retainedBytesMax": 8560 + }, + { + "library": "csharpmath", + "metric": "csharpmath-parse-batch", + "samples": 3, + "elapsedUsMedian": 27.630943333333335, + "elapsedUsMin": 22.85552666666667, + "elapsedUsMax": 36.966976666666667, + "allocatedBytesMedian": 5840.2616, + "allocatedBytesMin": 5836.7885333333334, + "allocatedBytesMax": 5846.252, + "retainedBytesMedian": 1728, + "retainedBytesMin": 1728, + "retainedBytesMax": 9952 + }, + { + "library": "csharpmath", + "metric": "csharpmath-layout-batch", + "samples": 3, + "elapsedUsMedian": 69.907566666666668, + "elapsedUsMin": 65.629046666666667, + "elapsedUsMax": 100.30413333333334, + "allocatedBytesMedian": 32511.4224, + "allocatedBytesMin": 32491.0944, + "allocatedBytesMax": 32569.983466666668, + "retainedBytesMedian": 0, + "retainedBytesMin": 0, + "retainedBytesMax": 6144 + }, + { + "library": "csharpmath", + "metric": "csharpmath-draw-batch", + "samples": 3, + "elapsedUsMedian": 182.39958333333334, + "elapsedUsMin": 154.53925, + "elapsedUsMax": 235.23773333333332, + "allocatedBytesMedian": 10336.569333333333, + "allocatedBytesMin": 10334.602666666668, + "allocatedBytesMax": 10336.633333333333, + "retainedBytesMedian": 10032, + "retainedBytesMin": 9880, + "retainedBytesMax": 11960 + }, + { + "library": "csharpmath", + "metric": "csharpmath-editor-x2", + "samples": 3, + "elapsedUsMedian": 39.2974, + "elapsedUsMin": 27.71045, + "elapsedUsMax": 50.9647, + "allocatedBytesMedian": 25914.764, + "allocatedBytesMin": 25914.456, + "allocatedBytesMax": 26034.764, + "retainedBytesMedian": 4496, + "retainedBytesMin": -4008, + "retainedBytesMax": 4496 + } +] diff --git a/research/ModeMathStudyBench/capture.ps1 b/research/ModeMathStudyBench/capture.ps1 new file mode 100644 index 00000000..4ce56e31 --- /dev/null +++ b/research/ModeMathStudyBench/capture.ps1 @@ -0,0 +1,78 @@ +param( + [Parameter(Mandatory = $true)][string]$ModeMathRoot, + [Parameter(Mandatory = $true)][string]$PackageDirectory, + [string]$Output = "$PSScriptRoot\raw-results.ndjson" +) +$ErrorActionPreference = 'Stop' +$project = Join-Path $PSScriptRoot 'ModeMathStudyBench.fsproj' +$modeMathCommit = '1ba976e0125fa03f41d6fe4f5aa4c60df89cfb10' +$csharpMathCommit = 'e1cb9d6270289ec1903022ba7a3275ba52a3d819' +$modeMathVersion = '0.12.0-csharpmath-study.1ba976e' +$modeMathPackageFile = "ModeMath.$modeMathVersion.nupkg" +$modeMathPackageSha256 = '0B03977732F2CE13EC9A290DD28A30DAFEF48E81ED36FC0B38670B4342700926' +$modeMathGit = Join-Path $ModeMathRoot '.git' +if (-not (Test-Path $modeMathGit)) { throw "ModeMathRoot is not a Git checkout: $ModeMathRoot" } +$modeMathRevision = (git -C $ModeMathRoot rev-parse HEAD).Trim() +if ($modeMathRevision -ne $modeMathCommit) { throw "ModeMathRoot must be pinned to $modeMathCommit (found $modeMathRevision)" } +if ((git -C $ModeMathRoot status --porcelain).Trim()) { throw 'ModeMathRoot must have a clean working tree' } +$csharpRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..')).Path +$csharpRevision = (git -C $csharpRoot rev-parse HEAD).Trim() +if ($csharpRevision -ne $csharpMathCommit) { throw "CSharpMath checkout must be pinned to $csharpMathCommit (found $csharpRevision)" } +git -C $csharpRoot diff --quiet HEAD -- CSharpMath CSharpMath.Rendering CSharpMath.SkiaSharp research/ModeMathStudyBench/ModeMathStudyBench.fsproj research/ModeMathStudyBench/Program.fs research/ModeMathStudyBench/capture.ps1 research/ModeMathStudyBench/aggregate.ps1 +if ($LASTEXITCODE -ne 0) { throw 'CSharpMath product/harness sources must be clean' } +git -C $csharpRoot diff --cached --quiet HEAD -- CSharpMath CSharpMath.Rendering CSharpMath.SkiaSharp research/ModeMathStudyBench/ModeMathStudyBench.fsproj research/ModeMathStudyBench/Program.fs research/ModeMathStudyBench/capture.ps1 research/ModeMathStudyBench/aggregate.ps1 +if ($LASTEXITCODE -ne 0) { throw 'CSharpMath product/harness sources must be clean in the index' } +$package = Get-Item (Join-Path $PackageDirectory $modeMathPackageFile) -ErrorAction SilentlyContinue +if ($null -eq $package) { throw "Missing pinned ModeMath package: $modeMathPackageFile" } +$packageHash = (Get-FileHash $package.FullName -Algorithm SHA256).Hash +if ($packageHash -ne $modeMathPackageSha256) { throw "Pinned package hash mismatch: expected $modeMathPackageSha256, found $packageHash" } +$restorePackagesPath = Join-Path ([IO.Path]::GetTempPath()) ("csharpmath-modemath-restore-" + [guid]::NewGuid().ToString('N')) +New-Item -ItemType Directory -Path $restorePackagesPath | Out-Null +try { + # Use a fresh package cache so an existing global cache cannot satisfy this study. + $configPath = Join-Path $restorePackagesPath 'NuGet.Config' + $localSource = [System.Security.SecurityElement]::Escape((Resolve-Path $PackageDirectory).Path) + $config = '' + Set-Content -LiteralPath $configPath -Encoding utf8 -Value $config + & dotnet restore $project --configfile $configPath --no-cache --force-evaluate -p:RestorePackagesPath=$restorePackagesPath -p:NoWarn=NU1605 + if ($LASTEXITCODE -ne 0) { throw 'Harness restore failed' } + $assetsPath = Join-Path $PSScriptRoot 'obj\project.assets.json' + $assets = Get-Content $assetsPath -Raw | ConvertFrom-Json + $assetLibraries = @($assets.libraries.psobject.Properties.Name) + if ($assetLibraries -notcontains "ModeMath/$modeMathVersion") { throw "Resolved assets do not contain pinned ModeMath/$modeMathVersion" } + $restoredPackage = Get-Item (Join-Path $restorePackagesPath "modemath\$modeMathVersion\$modeMathPackageFile") -ErrorAction SilentlyContinue + if ($null -eq $restoredPackage) { throw 'Pinned package was not restored into the isolated package cache' } + if ((Get-FileHash $restoredPackage.FullName -Algorithm SHA256).Hash -ne $modeMathPackageSha256) { throw 'Restored package hash does not match validated source package' } + & dotnet build $project -c Release --no-restore -p:RestorePackagesPath=$restorePackagesPath -p:GeneratePackageOnBuild=false + if ($LASTEXITCODE -ne 0) { throw 'Harness Release build failed' } +$runtime = ((& dotnet --list-runtimes | Select-String 'Microsoft.NETCore.App 10\.').Line -split ' ')[1] +$meta = [ordered]@{ record = 'metadata'; modeMathRevision = $modeMathRevision; csharpMathRevision = $csharpRevision; os = [Environment]::OSVersion.VersionString; runtime = $runtime; sdk = (& dotnet --version).Trim(); rid = 'win-x64'; configuration = 'Release'; font = 'ModeMath Layout 32 px / CSharpMath FontSize 24 pt at 96 DPI'; corpus = @('x^2 + y_1', '\frac{-b \pm \sqrt{b^2 - 4ac}}{2a}', '\int_0^1 x^2\,dx'); iterations = @{ parse = 10000; layout = 5000; draw = 2000; editor = 2000 }; packageFile = $package.Name; packageSha256 = $packageHash } +Set-Content -Path $Output -Value ($meta | ConvertTo-Json -Compress) -Encoding utf8 +function Invoke-Capture($library, $mode, $sample) { + $lines = & dotnet run --project $project -c Release --no-build -- $mode 2>&1 + if ($LASTEXITCODE -ne 0) { throw "capture failed: $library $mode" } + $expectedNames = if ($mode -like 'cold-*') { @("$library-cold-render") } else { @("$library-parse-batch", "$library-layout-batch", "$library-draw-batch", "$library-editor-x2") } + $rows = @() + foreach ($line in $lines) { + if (-not [string]::IsNullOrWhiteSpace($line) -and $line -notmatch '^\s*\{') { throw "Unexpected non-JSON output from $library $mode" } + if ($line -match '^\{') { + $row = $line | ConvertFrom-Json + $rows += $row + $row | Add-Member -NotePropertyName record -NotePropertyValue 'metric' + $row | Add-Member -NotePropertyName library -NotePropertyValue $library + $row | Add-Member -NotePropertyName mode -NotePropertyValue $mode + $row | Add-Member -NotePropertyName sample -NotePropertyValue $sample + ($row | ConvertTo-Json -Compress) | Add-Content -Path $Output -Encoding utf8 + } + } + if ($rows.Count -ne $expectedNames.Count) { throw "Expected $($expectedNames.Count) metric row(s) from $library $mode; found $($rows.Count)" } + $actualNames = @($rows | ForEach-Object { [string]$_.name } | Sort-Object -Unique) + if ((Compare-Object $actualNames ($expectedNames | Sort-Object -Unique))) { throw "Unexpected metric names from $library $mode" } +} +1..10 | ForEach-Object { $i = $_; Invoke-Capture 'modemath' 'cold-modemath' $i; Invoke-Capture 'csharpmath' 'cold-csharpmath' $i } +1..3 | ForEach-Object { $i = $_; Invoke-Capture 'modemath' 'steady-modemath' $i; Invoke-Capture 'csharpmath' 'steady-csharpmath' $i } + Write-Host "Captured $Output" +} +finally { + if (Test-Path $restorePackagesPath) { Remove-Item -LiteralPath $restorePackagesPath -Recurse -Force } +} diff --git a/research/ModeMathStudyBench/raw-results.ndjson b/research/ModeMathStudyBench/raw-results.ndjson new file mode 100644 index 00000000..a2d3c37a --- /dev/null +++ b/research/ModeMathStudyBench/raw-results.ndjson @@ -0,0 +1,45 @@ +{"record":"metadata","modeMathRevision":"1ba976e0125fa03f41d6fe4f5aa4c60df89cfb10","csharpMathRevision":"e1cb9d6270289ec1903022ba7a3275ba52a3d819","os":"Microsoft Windows NT 10.0.19045.0","runtime":"10.0.10","sdk":"10.0.302","rid":"win-x64","configuration":"Release","font":"ModeMath Layout 32 px / CSharpMath FontSize 24 pt at 96 DPI","corpus":["x^2 + y_1","\\frac{-b \\pm \\sqrt{b^2 - 4ac}}{2a}","\\int_0^1 x^2\\,dx"],"iterations":{"layout":5000,"parse":10000,"draw":2000,"editor":2000},"packageFile":"ModeMath.0.12.0-csharpmath-study.1ba976e.nupkg","packageSha256":"0B03977732F2CE13EC9A290DD28A30DAFEF48E81ED36FC0B38670B4342700926"} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":310606000,"allocatedBytes":270896,"retainedBytes":90192,"gen0":3,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"cold-modemath","sample":1} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":367005400,"allocatedBytes":22053456,"retainedBytes":3890800,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":1} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":190906200,"allocatedBytes":264704,"retainedBytes":87120,"gen0":3,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"cold-modemath","sample":2} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":513033600,"allocatedBytes":22065792,"retainedBytes":3896944,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":2} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":179948200,"allocatedBytes":264704,"retainedBytes":95344,"gen0":3,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"cold-modemath","sample":3} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":464426700,"allocatedBytes":22053456,"retainedBytes":3890800,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":3} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":216897200,"allocatedBytes":270896,"retainedBytes":90192,"gen0":3,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"cold-modemath","sample":4} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":675638500,"allocatedBytes":22065792,"retainedBytes":3905168,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":4} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":215197400,"allocatedBytes":264704,"retainedBytes":95344,"gen0":4,"gen1":4,"gen2":4,"record":"metric","library":"modemath","mode":"cold-modemath","sample":5} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":438885900,"allocatedBytes":22065792,"retainedBytes":3896944,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":5} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":208641100,"allocatedBytes":270896,"retainedBytes":98416,"gen0":3,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"cold-modemath","sample":6} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":428682700,"allocatedBytes":22053456,"retainedBytes":3890800,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":6} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":242464400,"allocatedBytes":264704,"retainedBytes":87120,"gen0":3,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"cold-modemath","sample":7} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":514501000,"allocatedBytes":22053456,"retainedBytes":3890800,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":7} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":249118500,"allocatedBytes":270896,"retainedBytes":90192,"gen0":5,"gen1":5,"gen2":5,"record":"metric","library":"modemath","mode":"cold-modemath","sample":8} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":559377100,"allocatedBytes":22053456,"retainedBytes":3899024,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":8} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":401301200,"allocatedBytes":264704,"retainedBytes":87120,"gen0":3,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"cold-modemath","sample":9} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":737039100,"allocatedBytes":22053456,"retainedBytes":3899024,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":9} +{"name":"modemath-cold-render","iterations":1,"elapsedNanoseconds":296254400,"allocatedBytes":270896,"retainedBytes":90192,"gen0":3,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"cold-modemath","sample":10} +{"name":"csharpmath-cold-render","iterations":1,"elapsedNanoseconds":578112800,"allocatedBytes":22065792,"retainedBytes":3896944,"gen0":6,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"cold-csharpmath","sample":10} +{"name":"modemath-parse-batch","iterations":10000,"elapsedNanoseconds":303523300,"allocatedBytes":87040040,"retainedBytes":8600,"gen0":21,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":1} +{"name":"modemath-layout-batch","iterations":5000,"elapsedNanoseconds":270446400,"allocatedBytes":102646232,"retainedBytes":-5152,"gen0":24,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":1} +{"name":"modemath-draw-batch","iterations":2000,"elapsedNanoseconds":214412300,"allocatedBytes":9746744,"retainedBytes":1312,"gen0":5,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":1} +{"name":"modemath-editor-x2","iterations":2000,"elapsedNanoseconds":45331000,"allocatedBytes":13840712,"retainedBytes":8560,"gen0":5,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":1} +{"name":"csharpmath-parse-batch","iterations":10000,"elapsedNanoseconds":828928300,"allocatedBytes":175387560,"retainedBytes":1728,"gen0":39,"gen1":3,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":1} +{"name":"csharpmath-layout-batch","iterations":5000,"elapsedNanoseconds":1504562000,"allocatedBytes":487671336,"retainedBytes":0,"gen0":106,"gen1":8,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":1} +{"name":"csharpmath-draw-batch","iterations":2000,"elapsedNanoseconds":1411426400,"allocatedBytes":62019800,"retainedBytes":9880,"gen0":16,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":1} +{"name":"csharpmath-editor-x2","iterations":2000,"elapsedNanoseconds":101929400,"allocatedBytes":51828912,"retainedBytes":-4008,"gen0":14,"gen1":3,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":1} +{"name":"modemath-parse-batch","iterations":10000,"elapsedNanoseconds":346684300,"allocatedBytes":87040040,"retainedBytes":376,"gen0":22,"gen1":4,"gen2":4,"record":"metric","library":"modemath","mode":"steady-modemath","sample":2} +{"name":"modemath-layout-batch","iterations":5000,"elapsedNanoseconds":343715400,"allocatedBytes":102640376,"retainedBytes":0,"gen0":24,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":2} +{"name":"modemath-draw-batch","iterations":2000,"elapsedNanoseconds":231709500,"allocatedBytes":9746408,"retainedBytes":-6912,"gen0":5,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":2} +{"name":"modemath-editor-x2","iterations":2000,"elapsedNanoseconds":98382100,"allocatedBytes":13840376,"retainedBytes":336,"gen0":5,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":2} +{"name":"csharpmath-parse-batch","iterations":10000,"elapsedNanoseconds":1109009300,"allocatedBytes":175103656,"retainedBytes":1728,"gen0":39,"gen1":3,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":2} +{"name":"csharpmath-layout-batch","iterations":5000,"elapsedNanoseconds":984435700,"allocatedBytes":487366416,"retainedBytes":6144,"gen0":106,"gen1":8,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":2} +{"name":"csharpmath-draw-batch","iterations":2000,"elapsedNanoseconds":1094397500,"allocatedBytes":62007616,"retainedBytes":11960,"gen0":16,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":2} +{"name":"csharpmath-editor-x2","iterations":2000,"elapsedNanoseconds":78594800,"allocatedBytes":51829528,"retainedBytes":4496,"gen0":14,"gen1":3,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":2} +{"name":"modemath-parse-batch","iterations":10000,"elapsedNanoseconds":219773300,"allocatedBytes":87040040,"retainedBytes":376,"gen0":21,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":3} +{"name":"modemath-layout-batch","iterations":5000,"elapsedNanoseconds":214164300,"allocatedBytes":102640400,"retainedBytes":24,"gen0":24,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":3} +{"name":"modemath-draw-batch","iterations":2000,"elapsedNanoseconds":203565800,"allocatedBytes":9746744,"retainedBytes":1312,"gen0":5,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":3} +{"name":"modemath-editor-x2","iterations":2000,"elapsedNanoseconds":33232900,"allocatedBytes":13840712,"retainedBytes":336,"gen0":5,"gen1":3,"gen2":3,"record":"metric","library":"modemath","mode":"steady-modemath","sample":3} +{"name":"csharpmath-parse-batch","iterations":10000,"elapsedNanoseconds":685665800,"allocatedBytes":175207848,"retainedBytes":9952,"gen0":39,"gen1":3,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":3} +{"name":"csharpmath-layout-batch","iterations":5000,"elapsedNanoseconds":1048613500,"allocatedBytes":488549752,"retainedBytes":0,"gen0":106,"gen1":8,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":3} +{"name":"csharpmath-draw-batch","iterations":2000,"elapsedNanoseconds":927235500,"allocatedBytes":62019416,"retainedBytes":10032,"gen0":16,"gen1":4,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":3} +{"name":"csharpmath-editor-x2","iterations":2000,"elapsedNanoseconds":55420900,"allocatedBytes":52069528,"retainedBytes":4496,"gen0":14,"gen1":3,"gen2":3,"record":"metric","library":"csharpmath","mode":"steady-csharpmath","sample":3} diff --git a/research/ModeMathStudyBench/results-captured.md b/research/ModeMathStudyBench/results-captured.md new file mode 100644 index 00000000..9245b823 --- /dev/null +++ b/research/ModeMathStudyBench/results-captured.md @@ -0,0 +1,15 @@ +# Captured benchmark aggregate + +This aggregate is computed from `raw-results.ndjson`: 10 fresh cold samples per library and 3 steady samples per library (each steady metric has 3 records). Values are medians; ranges are shown for elapsed time. The capture used ModeMath 32 px and CSharpMath 24 pt at 96 DPI. + +| Operation | ModeMath | CSharpMath | +|---|---:|---:| +| Cold render (ms; range) | 229.681 (179.948-401.301) | 513.767 (367.005-737.039) | +| Cold managed allocation (B; median) | 267,800 | 22,053,456 | +| Cold retained (B; median) | 90,192 | 3,896,944 | +| Parse, steady per formula (us / alloc) | 10.117 (7.326-11.556) / 2.833 KB | 27.631 (22.856-36.967) / 5.703 KB | +| Layout, steady per formula (us / alloc) | 18.030 (14.278-22.914) / 6.682 KB | 69.908 (65.629-100.304) / 31.749 KB | +| Draw, steady per formula (us / alloc) | 35.735 (33.928-38.618) / 1.586 KB | 182.400 (154.539-235.238) / 10.094 KB | +| Editor, 3-key sequence (us / alloc) | 22.666 (16.616-49.191) / 6.758 KB | 39.297 (27.710-50.965) / 25.307 KB | + +The normalized capture has 10 cold and 3 steady samples per library. Results are directional/manual, managed-only, and use different editor histories/timers; they are not a performance gate. Skia native allocations, JIT/code pages, process startup, and platform variance are outside these numbers.