Skip to content

docs(papers): verified citations for the three latex trees #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ronaldtse merged 1 commit into from
Sep 5, 2026
Merged

docs(papers): verified citations for the three latex trees #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified docs/paper-arabic/main.pdf
View file
Open in desktop
Binary file not shown.
17 changes: 11 additions & 6 deletions docs/paper-arabic/main.tex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ \section{Introduction}
$\approx$10M parameters) that \textbf{beats Sadeed on
SadeedDiac-25 with Misraj's own evaluator}: 3.25\% DER (CE) /
1.81\% (w/o CE) vs 7.29\% / 5.26\%, at 1/150th the parameters;
a 580M ByT5-base extends this to 2.81\% / 1.69\%,
a 580M ByT5-base~\cite{byt5} extends this to 2.81\% / 1.69\%,
beating Gemini-Flash-2.0 on all four metrics under a zero-skip
windowed protocol.
\item A data ablation showing in-domain DER falls 2.42\% $\rightarrow$ 0.99\% as
Expand Down Expand Up @@ -225,19 +225,19 @@ \subsection{The distilled client tier}
The same benchmark disciplines a 300M distilled student shipped for
browser/edge inference. From the 1.0 rung (8.26\%), a pre-registered
lever ladder reached \textbf{4.82\%} (42\% error reduction at
identical architecture and artifact size): the Muon optimizer alone
identical architecture and artifact size): the Muon optimizer~\cite{muon} alone
contributes $-2.96$pp (a controlled 2$\times$2 factorial closes
additively: optimizer $-2.96$, memory-layer capacity $-0.70$,
combined $-3.43$), fresher r7 teacher labels $-0.47$pp, and longer
training a further $-0.25$pp (4.57\%). Four registered negatives
bracket the ladder: multi-token-prediction as a training auxiliary
bracket the ladder: multi-token-prediction~\cite{mtp} as a training auxiliary
scored 5.09\% ($+0.26$pp vs control, with a disclosed preemption
confound); swapping news-domain training units for classical
Tashkeela at constant budget scored 5.81\% ($+0.98$pp); adding
18k classical units on top at matched epochs scored 4.82\%---flat
against the 3-epoch control and 0.25pp behind news-only training at
the same epochs, the add canceling the epoch gain entirely; and
on-policy generalized distillation (reverse-KL on student-sampled
on-policy generalized distillation~\cite{gkd} (reverse-KL on student-sampled
sequences scored by the frozen teacher) scored 6.00\% ($+1.18$pp,
the worst rung measured)---the domain-coverage hypothesis failing
in every direction it was tested. Together with the RL negatives,
Expand All @@ -260,8 +260,8 @@ \subsection{RL at SFT convergence: three negative results}
G2P the best checkpoint \emph{lost} 2 points on SentenceBench
homographs (77.34$\to$75.37\%), with dev reward degrading
monotonically past step 200.
\item \textbf{GTPO-GRPO} (entropy-weighted per-token credit,
arXiv:2508.04349; graded alignment-based reward): dev curve
\item \textbf{GTPO-GRPO} (entropy-weighted per-token
credit~\cite{gtpo}; graded alignment-based reward): dev curve
\emph{exactly} flat (5.9692\% at steps 0/100/150); the final
benchmark equals the teacher within protocol noise.
\end{itemize}
Expand Down Expand Up @@ -318,6 +318,11 @@ \section{Reproducibility}

\begin{thebibliography}{9}
\bibitem{sadeed} Sadeed: Arabic Diacritization. arXiv:2504.21635, 2025.
\bibitem{byt5} Xue et al. ByT5: Towards a token-free future with pre-trained byte-to-byte models. TACL, 2022. arXiv:2105.13626.
\bibitem{muon} Liu et al. Muon is Scalable for LLM Training. arXiv:2502.16982, 2025.
\bibitem{mtp} Gloeckle et al. Better \& Faster Large Language Models via Multi-token Prediction. arXiv:2404.19737, 2024.
\bibitem{gkd} Agarwal et al. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. ICLR, 2024. arXiv:2306.13649.
\bibitem{gtpo} Tan et al. GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy. arXiv:2508.04349, 2025.
\end{thebibliography}

\end{document}
Binary file modified docs/paper-hebrew/main.pdf
View file
Open in desktop
Binary file not shown.
12 changes: 9 additions & 3 deletions docs/paper-hebrew/main.tex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
ACL-2020 test corpora the same model is strong across modern, poetry,
and rabbinic line-level registers (0.52/0.24/0.45 DER), localizing the
weak surface specifically to paragraph-level Biblical text---register
coverage, not a general modern-text gap; we note D-Nikud, the recent
coverage, not a general modern-text gap; we note D-Nikud~\cite{dnikud}, the recent
open TavBERT+Bi-LSTM system, publishes no numbers on these corpora.
Error analysis shows the vowel-pointing accuracy of our model at
aligned positions is 97.1\%,
Expand Down Expand Up @@ -204,8 +204,14 @@ \section{Reproducibility}

\begin{thebibliography}{9}
\bibitem{dictabert} DictaBERT-large-char-menaked. Dicta, 2023--2024.
\bibitem{nakdimon} Elazar et al. Nakdimon. 2020.
\bibitem{byt5} Xue et al. ByT5: Towards a Token-Free Future. 2022.
\bibitem{nakdimon} Elazar Gershuni and Yuval Pinter. Restoring Hebrew
Diacritics Without a Dictionary (Nakdimon). Findings of NAACL, 2022.
arXiv:2105.05209.
\bibitem{byt5} Xue et al. ByT5: Towards a token-free future with
pre-trained byte-to-byte models. TACL, 2022. arXiv:2105.13626.
\bibitem{dnikud} Adi Rosenthal and Nadav Shaked. D-Nikud: Enhancing
Hebrew Diacritization with LSTM and Pretrained Models. arXiv:2402.00075,
2024.
\end{thebibliography}

\end{document}
Binary file modified docs/paper-umbrella/main.pdf
View file
Open in desktop
Binary file not shown.
13 changes: 10 additions & 3 deletions docs/paper-umbrella/main.tex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ \section{Why phonology for transliteration?}
vocalization that makes interscript.org's maps deterministic.

\section{System and results}
Common framework: ByT5 seq2seq (byte-level; tokenizer = UTF-8 bytes, so
Common framework: ByT5~\cite{byt5} seq2seq (byte-level; tokenizer = UTF-8 bytes, so
TS inference needs no vocabulary assets) except Arabic, where a compact
30M char-level encoder won on cost.

Expand All @@ -69,8 +69,8 @@ \section{System and results}
\toprule
Language & Task & Result & Published reference \\
\midrule
Arabic & diacritization & \textbf{2.29/1.33\% DER} & Sadeed 7.29/5.26\%, Gemini 3.19/2.38\%, GLM-5.3-Flash 8.80 (same benchmark+evaluator) \\
Hebrew & diacritization & \textbf{16.43\% DER} (Biblical) & DictaBERT 35.6\% (same test) \\
Arabic & diacritization & \textbf{2.29/1.33\% DER} & Sadeed~\cite{sadeed} 7.29/5.26\%, Gemini 3.19/2.38\%, GLM-5.3-Flash 8.80 (same benchmark+evaluator) \\
Hebrew & diacritization & \textbf{16.43\% DER} (Biblical) & DictaBERT~\cite{dictabert} 35.6\% (same test) \\
Thai & G2P & \textbf{2.32\% PER} & baseline 6.37\% (same test) \\
Persian & G2P/HA & \textbf{89.5\%} / 77.3\% (SB) & Homo-GE2PE 76.9\% \\
Persian & diacritization & \textbf{0.52\% CER} & (none existed) \\
Expand Down Expand Up @@ -151,4 +151,11 @@ \section{Reproducibility}
evaluation harnesses, \texttt{docs/RESULTS.md} ground truth, and the
five companion papers in \texttt{docs/paper-*/}.

\begin{thebibliography}{9}
\bibitem{sadeed} Sadeed: Arabic Diacritization. arXiv:2504.21635, 2025.
\bibitem{byt5} Xue et al. ByT5: Towards a token-free future with
pre-trained byte-to-byte models. TACL, 2022. arXiv:2105.13626.
\bibitem{dictabert} DictaBERT-large-char-menaked. Dicta, 2023--2024.
\end{thebibliography}

\end{document}