말뭉치 5개 판본 확대와 점역 정확도 97.38% - #194
Open
owjs3901 wants to merge 8 commits into
Open
Conversation
added 7 commits
September 8, 2026 21:25
Encoder changes, each grounded in the 2024 규정 PDF and measured against the five NIKL corpus editions (467,121 sentences): - 제51항: split after a colon that separates a label from its content, keep [다만 2] contrast pairs attached (+96) - 제52항 [붙임 1]: treat A/S, B/C as one Roman section (+66) - 제34/54항: keep a numeric parenthetical attached to its Korean word out of the math engine (+67) - 제72항: follow print spacing for list markers instead of injecting a blank (+877) - 제27/49/50/61항: fold fullwidth and compatibility characters onto the forms the standard defines; this removes 1,236 hard encode failures - 한자: transcribe Han ideographs by their Korean reading, from the Unicode Unihan kHangul property, with a 옛한글 guard for 제19-27항 - UEB 10.6.2/10.7: spelling fallback for con and ever when the pronunciation source has no entry - 한글 맞춤법: attach a comma and a spaced 붙임표 to the word on the left Roman-section state is now centralized: production code no longer assigns is_english / eeds_english_continuation / oman_number_chain directly but goes through ules::roman_mode, which asserts that a section cannot be open while a continuation or a number bridge is pending. That invariant caught a dead flag combination, removed here. Tooling: - hree_way_bench reports braillify / 점자세상 / 점사랑 side by side - oman_marker_bench tracks the four Roman-marker error classes apart, since they cancel out in an aggregate figure - etch-jeomsarang-lib replaces the GUI automation with a direct BrailleTransLibrary DLL call - corpus benches for both competitors questions/ records the 26 items the PDF alone cannot settle, with per-edition counts; several turn out to be conventions that differ between corpus editions rather than defects.
점자세상 is re-fetched from the API and 점사랑 now comes from the BrailleTransLibrary DLL instead of GUI automation, so both world and jeomsarang are regenerated across korean/, math/ and english/. These fields are read-only benchmark data: they are never compared against by test_by_testcase and never consulted by the encoder.
The NIKL parallel corpus now covers five editions, 467,121 sentences: 2021 93,115 2022 108,795 2023 126,693 2024 54,990 2025 83,528 Each record takes its latest `revision` as the reference, matching how 2024 and 2025 were imported. `rule_map.json` marks every edition `benchmark: true` so they report accuracy without entering pass/fail. `world` and `jeomsarang` are filled for every sentence: 점자세상 from the braillekorea API and 점사랑 from the BrailleTransLibrary DLL. 2025 was already tracked and is only refreshed here. The 2021 shards are split at 12,000 records rather than 25,000; its sentences are long enough that the default chunk produced a 97 MB file.
Cluster the word-level mismatches by cell-diff signature and keep only the one-directional ones: if the reverse signature is absent, the reference is consistent and the defect is ours. - UEB 10.7 `ever`: a base with an inflectional `s` (`Clevers`) is still the unstressed `-er` unit when the word is absent from CMUdict. - 61: U+2019 between Roman letters is the apostrophe, not the Rule 54 closing quote, so it stays inside the Roman section (`I’m`). - UEB 10.7 `some`: a base ending in the combining-form linking vowel `o` (`exo`/`azoto`) exposes the word-final morpheme without a dictionary. - 8/10: a jamo that starts the word is a leading bullet, not part of the following word; Rule 10 covers jamo embedded in a bracketed pronunciation. Corpus 454,792 -> 454,864 (97.38%); rule fixtures stay 5,141/5,141 and the Roman marker counts stay 806/64/300/369. Record the clusters that the rules cannot settle as questions 27-29: UEB contractions inside all-caps words (10.12.1 `POW`), `ever` in words absent from the dictionary (against the `eversion`/`reverify` examples), and the capital indicators of mixed-case words (8.8.2 `KBr`).
Two more one-directional clusters, both from a Korean rule swallowing a character that belongs to the Roman run. - 73: the underscore is a blank to be filled in, so it is written as the Korean marker. Between two Roman letters (`VR_S`) it is instead the low line inside a Rule 32 Roman name. A digit on the right is left alone: it may be a math subscript (`\int_0`). - 49: the full stop of a numbered item (`4.PMP`, `30.FC`) belongs to the digit before it, not to the Roman word after it. Routing it through the Roman run emitted the Roman indicator ahead of the stop and reversed the two cells. Corpus 454,864 -> 454,899 (97.38%); rule fixtures stay 5,141/5,141 and the Roman marker counts stay 806/64/300/369.
`ABCabc123` now takes the UEB 8.6.3 capitals word indicator and terminator around its capitalised part instead of one capital indicator per letter.
The parallel corpus is no longer one `corpus` directory: it is five per-edition directories from 2021 to 2025. Sum the testcase report over every edition and give each one its own row, and list them separately in the landing test-case browser.
Contributor
Braillify testcase report
Command: |
The questions for the National Institute of Korean Language are still being trimmed, so they do not belong in this pull request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
요약
국립국어원 한국어-한국점자 병렬 말뭉치를 5개 판본 467,121문장으로 넓히고,
2024 개정 한국 점자 규정과 통일영어점자(UEB)에 근거해 점역 정확도를 끌어올렸습니다.
규정만으로 정할 수 없는 항목은 구현하지 않고
questions/에 근거와 함께 남겼습니다.규정 fixture 는 5,141 / 5,141 (100%) 입니다. 같은 fixture 에서 점자세상은
44.38%, 점사랑은 35.52% 입니다.
판본별 정확도
말뭉치 확대
test_cases/corpus하나였던 것을test_cases/{2021,2022,2023,2024,2025}_corpus다섯 개로 나누고 2021~2023 판본을 새로 임포트했습니다. 말뭉치는
rule_map.json에서
benchmark: true라 pass/fail 에 들어가지 않고 정확도만 보고합니다.전 문장에 경쟁사 점역 결과(
world,jeomsarang)를 채워 비교 기준을 만들었습니다.CI 의 testcase 보고와 랜딩의 테스트케이스 열람도 판본별로 나누어 보도록 맞췄습니다.
고친 규정
말뭉치 불일치를 셀 차이 서명으로 군집화한 뒤 역방향이 없는 것만 골랐습니다.
양방향인 군집은 판본·문맥에 따라 갈리는 항목이라 규정 회신 없이는 손대지 않았습니다.
청군:백군같은 대비쌍은 붙임A/S,B/C는 한 로마자 구간kHangul8,525자, 옛한글 4중 가드)con, 10.7ever·some— 사전에 없는 낱말의 철자 판정⠿4.PMP)은 앞 숫자의 것로마자 구간 상태 정리
로마자표
⠴·종료표⠲·연속표⠰를 결정하는 세 플래그를 프로덕션 코드10개 파일 88곳에서 직접 대입하고 있었습니다. 이를
rules/roman_mode.rs의동사 11개로 모두 모으고, 모순 조합을
debug_assert_consistent로 막았습니다.이 검증이 실제 위반 1건을 잡아냈고 그 조합은 죽은 상태로 확인되어 제거했습니다.
각 단계는 출력이 바이트 단위로 같은지 확인하며 진행했습니다.
안전망으로
roman_marker_bench를 추가했습니다. 표지 과잉·누락 네 가지는 서로반대 방향이라 총 정확도만 보면 한쪽을 고치며 다른 쪽을 망가뜨려도 드러나지
않습니다. 실제로 이 벤치가 −7,849, −7,475 회귀를 즉시 잡아 되돌렸습니다.
벤치마크
three_way_bench— braillify / 점자세상 / 점사랑 교차표roman_marker_bench— 로마자 표지 4유형 개별 추적BrailleTransLibraryDLL 을 직접 호출하도록 바꾸고,18가지 설정을 재어 가장 좋은 조합을 골랐습니다
bench/FINAL_BENCHMARK_COMPARISON.md국립국어원 질의
규정만으로 확정할 수 없거나 규정과 말뭉치가 어긋나는 항목 29건 을
questions/에 정리했습니다. 이번에 다음 세 건을 새로 넣었습니다.NOW,KBSTAR).UEB 10.12.1 의
POW ⠏⠕⠺와 짜임새가 같아 규정만으로는 못 뒤집습니다.ever약자 (Weverse124건).약자를 쓰면 말뭉치가 +93 이지만 규정 예제
eversion·reverify·Guinevere·Monteverdi네 개가 어긋나 되돌렸습니다.NOx,IPnomics,CO2,AFoCO).UEB 8.8.2 의
KBr·MHz와 동형이고, 참조 안에서도 기준이 갈립니다.검증
cargo test --release -p braillify— 4,684 + 541 통과, 실패 0cargo clippy --release -p braillify --all-targets -- -D warnings통과