Integrate the OCR sigma and summation fixes onto main

.github/workflows/package.yml

@@ -3,6 +3,7 @@ name: package
 on:
   push:
     branches:
+      - main
       - cursor/mathscrap-packaging-*
   pull_request:
     branches:

README.md

@@ -28,6 +28,10 @@ log_2(8)    log_{2}(8)    log₂(8)    log2(8)    log(2, 8)
 
 `log(2)` and `log 2` stay log₁₀(2). They are not “base 2”. `lg` does not take a base glued to the name: `lg2(3)` is lg(2)·3. The base must be greater than 0 and not 1, and the argument must be greater than 0. Otherwise there is no real value, and the plot does not draw a fake segment across a gap or an asymptote.
 
+解三角形时还会给出 X(91) 和 X(92)，分别是 X(47) 和 X(48) 的等角共轭。直角三角形里没有 X(92)。
+
+Solving a triangle also reports X(91) and X(92), the isogonal conjugates of X(47) and X(48). A right triangle has no X(92).
+
 ## 快速开始
 
 ## Quick start
@@ -50,7 +54,7 @@ The dev server is `http://127.0.0.1:47321`. `npm run preview` serves the product
 | `npm run dev` | 本机开发服务器，含 MCP 桥和 AI 转发。 / Local dev server, including the MCP bridge and the AI forwarder. |
 | `npm run build` | 类型检查并构建 `dist/`。 / Typecheck and build `dist/`. |
 | `npm run lint` | oxlint，有警告也失败。 / oxlint. Warnings fail the command. |
-| `npm test:pack` 或 `npm run test:i18n` | 打包元数据，以及中英词条、语言记忆。 / Packaging metadata, plus Chinese/English keys and saved language. |
+| `npm run test:pack` 或 `npm run test:i18n` | 打包元数据，以及中英词条、语言记忆。 / Packaging metadata, plus Chinese/English keys and saved language. |
 | `npm run smoke` | 用真实 WASM 检查数值、对数和图上有没有点。 / Real WASM: numbers, logarithms, and whether a curve has points. |
 | `npm run smoke:ocr` | 手写预处理和假的编码器/解码器，不加载 ONNX。 / Ink preprocess and a fake encoder/decoder. No ONNX. |
 | `npm run ocr:live` | 本机用仓库里的 CoMER INT8 读几笔样例。 / Run the bundled CoMER INT8 on a few sample strokes. |
@@ -106,6 +110,10 @@ Recognition is on by default. About a second after the ink settles, a text gate
 
 The model is CoMER (ECCV 2022) INT8 ONNX. The weights are distributed by [ink-on](https://github.com/kimseungdae/ink-on) under Apache-2.0. It can read some clear short formulas. It is not a complete recognizer for arbitrary handwriting: a miss leaves the ink, and it does not substitute `y = x`. A lone `sigma` is the symbol. Sums and integrals stay on the paper and do not open the coordinate view by themselves.
 
+模型把单独的 σ 读成 `0`、`Y` 或 `u`，而且笔迹本身像 σ 时，才改写成 `sigma`。公式、闭合的 0、u 形的碗和 Y 的分叉不会被改。求和号上下的两道横杠不当成字母上的箭头。
+
+A lone σ that the model reads as `0`, `Y`, or `u` is rewritten to `sigma` only when the ink itself looks like σ. A formula, a closed 0, a u-shaped bowl, and a Y fork are left alone. The two bars of a summation are not treated as an arrow over a letter.
+
 | 文件 File | 内容 / Contents |
 | --- | --- |
 | `public/models/comer/encoder_int8.onnx` | 编码器，约 3.4 MB。 / Encoder, about 3.4 MB. |
@@ -168,9 +176,9 @@ iOS 最低系统是 16.4。能装到别人手机上的 IPA 需要 Apple Develope
 
 The iOS minimum is 16.4. An IPA that installs on someone else's phone needs an Apple Developer certificate and profile: `IOS_CERTIFICATE`, `IOS_CERTIFICATE_PASSWORD`, `IOS_MOBILE_PROVISION`, or `APPLE_API_ISSUER`, `APPLE_API_KEY`, `APPLE_API_KEY_PATH`. This repository does not enroll a developer account, does not upload to TestFlight or the App Store, and has not been installed on an iPhone.
 
-`.github/workflows/package.yml` 里的 `test` 作业在 `ubuntu-latest` 上跑 `npm run lint`、`npm run test:rust`、`npm run smoke`、`npm run smoke:ocr`、`npm run smoke:ai`、`npm run test:pack`，然后拉起 dev 服务器，依次跑 `npm run e2e:sheet` 和 `npm run e2e`（逐条作图）。失败时上传 `e2e-shots`、`e2e-plots` 和 `dev-server.log`。这些步骤不需要密钥。推到 `cursor/mathscrap-packaging-*` 时，测试通过后同一 workflow 再构建网页、debug APK 和未签名的 Windows 安装程序。指向 `main` 的 pull request 只跑 `test`。`.github/workflows/package-ios.yml` 只在手动启动时使用 macOS runner。这些 workflow 不创建 GitHub Release。本文件不提供下载地址。
+`.github/workflows/package.yml` 在推送到 `main` 或 `cursor/mathscrap-packaging-*` 时，以及指向 `main` 的 pull request 上运行。`test` 作业没有额外的跳过条件：这三种情况都会在 `ubuntu-latest` 上跑 `npm run lint`、`npm run test:rust`、`npm run smoke`、`npm run smoke:ocr`、`npm run smoke:ai`、`npm run test:pack`，然后拉起 dev 服务器，依次跑 `npm run e2e:sheet` 和 `npm run e2e`（逐条作图）。失败时上传 `e2e-shots`、`e2e-plots` 和 `dev-server.log`。这些步骤不需要密钥。网页、debug APK 和未签名 Windows 安装程序仍只在 push 上、并且 `test` 通过之后构建；pull request 不打安装包。`.github/workflows/package-ios.yml` 只在手动启动时使用 macOS runner。这些 workflow 不创建 GitHub Release。本文件不提供下载地址。
 
-The `test` job in `.github/workflows/package.yml` runs on `ubuntu-latest`: `npm run lint`, `npm run test:rust`, `npm run smoke`, `npm run smoke:ocr`, `npm run smoke:ai`, `npm run test:pack`, then starts the dev server and runs `npm run e2e:sheet` and `npm run e2e` (one plot per formula). On failure it uploads `e2e-shots`, `e2e-plots`, and `dev-server.log`. Those steps need no secrets. A push to `cursor/mathscrap-packaging-*` builds the web bundle, a debug APK, and an unsigned Windows installer in the same workflow after that job passes. A pull request into `main` runs only `test`. `.github/workflows/package-ios.yml` uses a macOS runner only when started by hand. These workflows do not create a GitHub Release. This file does not publish a download URL.
+`.github/workflows/package.yml` runs on a push to `main` or `cursor/mathscrap-packaging-*`, and on a pull request into `main`. The `test` job has no extra skip condition, so all three of those runs execute it on `ubuntu-latest`: `npm run lint`, `npm run test:rust`, `npm run smoke`, `npm run smoke:ocr`, `npm run smoke:ai`, `npm run test:pack`, then a dev server, then `npm run e2e:sheet` and `npm run e2e` (one plot per formula). On failure it uploads `e2e-shots`, `e2e-plots`, and `dev-server.log`. Those steps need no secrets. The web bundle, debug APK, and unsigned Windows installer still build only on a push, and only after `test` passes. A pull request does not build installers. `.github/workflows/package-ios.yml` uses a macOS runner only when started by hand. These workflows do not create a GitHub Release. This file does not publish a download URL.
 
 ## 许可和作者
 

scripts/ocr-smoke.ts

@@ -14,11 +14,12 @@ import { readFileSync } from 'node:fs'
 import type { Stroke } from '../src/board/types.ts'
 import type { ComerSession, ComerTensor } from '../src/ocr/comer/decode.ts'
 import { bindInk, findOverArrow } from '../src/ocr/comer/bind.ts'
-import { classifyInk, looksLikeIsolatedSigma } from '../src/ocr/gate.ts'
+import { classifyInk, looksLikeIsolatedSigma, mathCues } from '../src/ocr/gate.ts'
 import { agreeConfidence, recognizeStrokes } from '../src/ocr/comer/engine.ts'
 import { OCR_ACCEPT } from '../src/ocr/threshold.ts'
 import { isInkMeaningful, preprocessStrokes, resamplePoints, type InkStroke } from '../src/ocr/comer/preprocess.ts'
-import { arrowLetter, inkFixtures, isolatedSigma, proseNote, sumBar, sumGlyph } from '../src/ocr/comer/samples.ts'
+import { arrowLetter, bowlU, closedZero, forkY, inkFixtures, isolatedSigma, proseNote, sigmaHook, sigmaLoop, sumBar, sumGlyph } from '../src/ocr/comer/samples.ts'
+import { looksLikeIsolatedSigma as sigmaGlyph, repairIsolatedSigma } from '../src/ocr/comer/sigma.ts'
 import { tokensToSource, vocabFromJson } from '../src/ocr/comer/source.ts'
 import { filterAbsurd, hasParenGeometry, recognizeInk } from '../src/ocr/inkShape.ts'
 
@@ -443,6 +444,13 @@ const boundVec = bindInk(arrowed, 'a')
 assert.match(boundVec, /^\\vec\{a\} = \([0-9.-]+, [0-9.-]+\)$/)
 assert.equal(bindInk(arrowed, 'a = (1, 2, 2)'), boundVec)
 assert.doesNotMatch(boundVec, /1,\s*2,\s*2/)
+assert.equal(findOverArrow(sumBar()), null, 'Σ bars are not an over-arrow')
+assert.equal(bindInk(sumBar(), 'a'), '')
+assert.equal(bindInk(sumBar(), '\\vec{a}'), '')
+assert.equal(bindInk(sumBar(), '\\sum'), '\\sum')
+assert.equal(bindInk(sumBar(), '\\sum_{i = 1}^{n} i'), '\\sum_{i = 1}^{n} i')
+assert.equal(agreeConfidence(sumBar(), '\\vec{a} = (1, 0)', 0.48), 0.48)
+assert.equal(agreeConfidence(sumBar(), '\\sum', 0.45), 0.78)
 assert.equal(bindInk(arrowed, '\\sum'), '\\sum')
 assert.equal(bindInk(arrowed, '\\sum_{i = 1}^{n} i'), '\\sum_{i = 1}^{n} i')
 assert.equal(bindInk(arrowed, 'y = 2'), 'y = 2')
@@ -501,6 +509,44 @@ assert.equal(plain[0]?.text, 'y = 2')
 assert.ok((plain[0]?.confidence ?? 1) < OCR_ACCEPT, `ordinary formula is not lifted (${plain[0]?.confidence})`)
 console.log('ocr agreement ok', { arrow: agreed[0], sum: sumAgreed[0], faint: faintArrow[0]?.confidence, plain: plain[0]?.confidence })
 
+const loop = sigmaLoop()
+const hook = sigmaHook()
+const zero = closedZero()
+const bowl = bowlU()
+const fork = forkY()
+assert.equal(sigmaGlyph(loop), true)
+assert.equal(sigmaGlyph(hook), true)
+assert.equal(sigmaGlyph(zero), false, 'a closed 0 is not σ')
+assert.equal(sigmaGlyph(bowl), false, 'a u bowl is not σ')
+assert.equal(sigmaGlyph(fork), false, 'a Y fork is not σ')
+assert.equal(sigmaGlyph(proseNote()), false)
+assert.equal(sigmaGlyph(y2strokes), false)
+assert.equal(repairIsolatedSigma(loop, '0'), 'sigma')
+assert.equal(repairIsolatedSigma(loop, 'Y'), 'sigma')
+assert.equal(repairIsolatedSigma(hook, 'u'), 'sigma')
+assert.equal(repairIsolatedSigma(loop, 'y = 0'), null, 'a formula is not rewritten')
+assert.equal(repairIsolatedSigma(zero, '0'), null)
+assert.equal(repairIsolatedSigma(bowl, 'u'), null)
+assert.equal(repairIsolatedSigma(fork, 'Y'), null)
+assert.equal(repairIsolatedSigma(proseNote(), '0'), null)
+for (const [ink, token] of [[loop, '0'], [hook, 'u'], [loop, 'Y']] as const) {
+  const read = await recognizeStrokes(ink, scripted([id(token)], weak), scripted([id(token)], weak), vocab, 1)
+  assert.equal(read[0]?.text, 'sigma', `${token} on σ → ${JSON.stringify(read)}`)
+  assert.ok((read[0]?.confidence ?? 0) >= OCR_ACCEPT, `${token} on σ confidence ${read[0]?.confidence}`)
+}
+const faintZero = await recognizeStrokes(loop, scripted([id('0')], faint), scripted([id('0')], faint), vocab, 1)
+assert.notEqual(faintZero[0]?.text, 'sigma', 'a faint 0 is not promoted to σ')
+assert.ok((faintZero[0]?.confidence ?? 1) < OCR_ACCEPT)
+const realZero = await recognizeStrokes(zero, scripted([id('0')], 6), scripted([id('0')], 6), vocab, 1)
+assert.equal(realZero[0]?.text, '0', JSON.stringify(realZero))
+const realU = await recognizeStrokes(bowl, scripted([id('u')], 6), scripted([id('u')], 6), vocab, 1)
+assert.ok(!realU.some((c) => c.text === 'sigma'), JSON.stringify(realU))
+const realY = await recognizeStrokes(fork, scripted([id('Y')], 6), scripted([id('Y')], 6), vocab, 1)
+assert.ok(!realY.some((c) => c.text === 'sigma'), JSON.stringify(realY))
+const formulaZero = await recognizeStrokes(loop, scripted([id('y'), id('='), id('0')], weak), scripted([id('y'), id('='), id('0')], weak), vocab, 1)
+assert.equal(formulaZero[0]?.text, 'y = 0', JSON.stringify(formulaZero))
+console.log('ocr sigma glyph ok')
+
 function sumLikeSigma(): InkStroke[] {
   return [
     {
@@ -523,9 +569,14 @@ function sumLikeSigma(): InkStroke[] {
 assert.equal(classifyInk(y2strokes), 'formula')
 assert.equal(classifyInk(arrowLetter()), 'formula')
 assert.equal(classifyInk(proseNote()), 'text', 'a line of letters is not a formula')
+assert.equal(classifyInk(sigmaLoop()), 'formula', 'an isolated σ still reaches the model')
+assert.equal(classifyInk(closedZero()), 'formula')
 assert.equal(classifyInk(sumLikeSigma()), 'formula', 'Σ with limits is a formula')
 assert.equal(classifyInk(sumGlyph()), 'formula')
 assert.equal(classifyInk(sumBar()), 'formula')
+assert.ok(mathCues(sumBar()).includes('sum'), 'barred Σ is a sum, not only an arrow')
+assert.ok(!mathCues(sumBar()).includes('arrow'))
+assert.ok(mathCues(arrowLetter()).includes('arrow'))
 assert.equal(classifyInk(inkFixtures(3).find((s) => s.name === 'frac')!.strokes), 'formula')
 console.log('ocr gate ok', { prose: classifyInk(proseNote()), formula: classifyInk(y2strokes) })
 assert.equal(tokensToSource(['\\ln', '(', 'x', ')']).replace(/\s+/g, ''), 'ln(x)')

scripts/plot-cases.ts

@@ -53,7 +53,8 @@ export const cases: [string, string, Expect][] = [
   ['直线过两点', 'y = kx + b, 过(1, 3), 过(2, 5)', { solve: { k: 2, b: 1 } }],
   // ---- triangles / coordinate geometry ----
   ['解三角形 SAS', '△ABC: a = 5, b = 7, C = 60°', { dim: 2, facts: ['\\sqrt{39}'], minFacts: 10 }],
-  ['解三角形 SSS', '△ABC: a=3, b=4, c=5', { dim: 2, facts: ['90', 'S_{\\text{海伦}}', '|OI|', 'r_a=', 'AF_{c}', '\\Pi_{\\text{Ceva}}', '\\Pi_{\\text{Menelaus}}', '\\text{Stewart}_{m_', 'ID_{a}', '\\mathcal{A}_', 'OG:GH', 'R_{\\text{九点}}', '|NI|', '|NI_a|', 'Sp', 'Na', 'IG:GNa'], minFacts: 45 }],
+  ['解三角形 SSS', '△ABC: a=3, b=4, c=5', { dim: 2, facts: ['90', 'S_{\\text{海伦}}', '|OI|', 'r_a=', 'AF_{c}', '\\Pi_{\\text{Ceva}}', '\\Pi_{\\text{Menelaus}}', '\\text{Stewart}_{m_', 'ID_{a}', '\\mathcal{A}_', 'OG:GH', 'R_{\\text{九点}}', '|NI|', '|NI_a|', 'Sp', 'Na', 'IG:GNa', 'Ge=', 'K=', '\\omega', '\\Omega', 'L=', 'Mi', 'Sc', 'Be=', 'H_a', 'X_a', 'r_{\\text{Sp}}', 'T_a', 'S_{\\text{Steiner}}', 'M_a', 'Ex=', 'S_{1}', 'St=', 'Ty=', 'N_1', 'Py', 'V_{\\text{out}}', 'X_{39}', 'Hf_0', 'Km', 'Ap', 'F_1', 'Fe', 'Ed', 'Ip', 'Mo', 'S_{+}', 'S_{-}', 'X_{30}', 'X_{57}', 'X_{58}', 'X_{84}', 'X_{12}=', 'X_{59}=', 'X_{60}=', 'Gr=', 'X_{81}=', 'X_{42}=', 'X_{65}=', 'X_{76}=', 'X_{86}=', 'X_{31}=', 'X_{32}=', 'X_{69}=', 'X_{36}', 'X_{41}', 'X_{48}', 'X_{20}=', 'X_{23}=', 'X_{35}=', 'X_{38}=', 'X_{61}=', 'X_{62}=', 'X_{63}=', 'X_{43}=', 'X_{44}=', 'X_{45}=', 'X_{46}=', 'X_{47}=', 'X_{49}=', 'X_{50}=', 'X_{79}=', 'X_{80}=', 'X_{51}=', 'X_{91}=', 'X_{70}=', 'X_{71}=', 'X_{73}='], minFacts: 170 }],
+  ['欧拉反射点（锐角）', '△ABC: a=6, b=7, c=8', { dim: 2, facts: ['E_r', 'E_r \\in (O)', 'Ko', 'Cl', 'Yff', 'X_{25}', 'O(H_a', 'X_{24}', 'X_{64}', 'X_{26}', 'X_{53}', 'X_{27}', 'X_{29}', 'X_{92}', 'X_{28}'], minFacts: 10 }],
   ['解三角形 SSA 两解', '△ABC: a = 6, b = 8, A = 40°', { dim: 2 }],
   ['坐标 + 垂直', 'A(0,0), B(4,0), C(1,t)\nAC ⊥ BC', { dim: 2, solve: { t: Math.sqrt(3) }, inView: [[0, 0], [4, 0]] }],
   ['平行四边形', 'A(0,0), B(4,0), C(5,3), D(1,3)', { dim: 2, facts: ['平行四边形'], inView: [[5, 3]] }],

src/ocr/comer/bind.ts

@@ -41,7 +41,38 @@ function arrowReading(squashed: string): boolean {
 }
 
 /** A flat stroke sitting above the rest of the ink, wider than the letter. */
+/**
+ * ∑ drawn as a top bar, a zigzag, and a bottom bar. The top bar is flat and
+ * high, so it used to count as an over-arrow and a letter reading became a vector.
+ */
+export function looksLikeSumBars(strokes: InkStroke[]): boolean {
+  const drawn = strokes.filter((s) => s.points.length >= 2)
+  if (drawn.length < 3) return false
+  const all = bounds(drawn.flatMap((s) => s.points))
+  if (all.h < 24 || all.w < 16) return false
+  const flats = drawn.filter((s) => {
+    const b = bounds(s.points)
+    return b.w > all.w * 0.55 && b.h < Math.max(8, b.w * 0.35)
+  })
+  if (flats.length < 2) return false
+  const boxes = flats.map((s) => bounds(s.points)).sort((a, b) => a.cy - b.cy)
+  const top = boxes[0]
+  const bot = boxes[boxes.length - 1]
+  if (top.cy > all.y0 + all.h * 0.4) return false
+  if (bot.cy < all.y0 + all.h * 0.6) return false
+  const gap = bot.y0 - top.y1
+  if (gap < all.h * 0.35) return false
+  const overlap = Math.min(top.x1, bot.x1) - Math.max(top.x0, bot.x0)
+  if (overlap < Math.min(top.w, bot.w) * 0.5) return false
+  return drawn.some((s) => {
+    if (flats.includes(s)) return false
+    const b = bounds(s.points)
+    return b.h > gap * 0.45 && b.y0 < top.y1 + gap * 0.35 && b.y1 > bot.y0 - gap * 0.35
+  })
+}
+
 export function findOverArrow(strokes: InkStroke[]): { x: number; y: number } | null {
+  if (looksLikeSumBars(strokes)) return null
   const drawn = strokes.filter((s) => s.points.length >= 2)
   if (drawn.length < 2) return null
   const all = bounds(drawn.flatMap((s) => s.points))

src/ocr/comer/engine.ts

@@ -2,6 +2,7 @@ import { bindInk, findOverArrow } from './bind.ts'
 import { decodeComer, type ComerSession, type ComerTensor } from './decode.ts'
 import { looksLikeIsolatedSigma } from '../gate.ts'
 import { isInkMeaningful, preprocessStrokes, type InkStroke } from './preprocess.ts'
+import { repairIsolatedSigma } from './sigma.ts'
 import { decodeTokens, tokensToSource, type Vocab } from './source.ts'
 
 export interface OcrCandidate {
@@ -64,14 +65,19 @@ export async function recognizeStrokes(strokes: InkStroke[], encoder: ComerSessi
   const out: OcrCandidate[] = []
   const seen = new Set<string>()
   for (const hit of hits) {
-    const latex = decodeTokens(hit.ids, vocab).join(' ')
-    const text = bindInk(strokes, tokensToSource(decodeTokens(hit.ids, vocab)))
-    if (!text || seen.has(text)) continue
-    seen.add(text)
+    const decoded = decodeTokens(hit.ids, vocab)
+    const latex = decoded.join(' ')
+    const source = tokensToSource(decoded)
     const n = Math.max(1, hit.ids.length)
     const mean = hit.logProb / n
     const model = Math.exp(mean)
     if (!Number.isFinite(model) || model < 0.2) continue
+    // A mid-or-better read of 0 / Y / u is σ only when the ink has that glyph.
+    // Fainter than AGREE_AT stays the model's text, so a weak guess is not promoted.
+    const repaired = model >= AGREE_AT ? repairIsolatedSigma(strokes, source) : null
+    const text = repaired ?? bindInk(strokes, source)
+    if (!text || seen.has(text)) continue
+    seen.add(text)
     out.push({ text, confidence: agreeConfidence(strokes, text, model), latex })
   }
   out.sort((a, b) => b.confidence - a.confidence || a.text.localeCompare(b.text))