diff --git a/docs/paper-arabic/main.pdf b/docs/paper-arabic/main.pdf index 6c5626b..989c904 100644 Binary files a/docs/paper-arabic/main.pdf and b/docs/paper-arabic/main.pdf differ diff --git a/docs/paper-arabic/main.tex b/docs/paper-arabic/main.tex index 3cc139f..2c86b96 100644 --- a/docs/paper-arabic/main.tex +++ b/docs/paper-arabic/main.tex @@ -58,7 +58,7 @@ \section{Introduction} $\approx$10M parameters) that \textbf{beats Sadeed on SadeedDiac-25 with Misraj's own evaluator}: 3.25\% DER (CE) / 1.81\% (w/o CE) vs 7.29\% / 5.26\%, at 1/150th the parameters; - a 580M ByT5-base extends this to 2.81\% / 1.69\%, + a 580M ByT5-base~\cite{byt5} extends this to 2.81\% / 1.69\%, beating Gemini-Flash-2.0 on all four metrics under a zero-skip windowed protocol. \item A data ablation showing in-domain DER falls 2.42\% $\rightarrow$ 0.99\% as @@ -225,19 +225,19 @@ \subsection{The distilled client tier} The same benchmark disciplines a 300M distilled student shipped for browser/edge inference. From the 1.0 rung (8.26\%), a pre-registered lever ladder reached \textbf{4.82\%} (42\% error reduction at -identical architecture and artifact size): the Muon optimizer alone +identical architecture and artifact size): the Muon optimizer~\cite{muon} alone contributes $-2.96$pp (a controlled 2$\times$2 factorial closes additively: optimizer $-2.96$, memory-layer capacity $-0.70$, combined $-3.43$), fresher r7 teacher labels $-0.47$pp, and longer training a further $-0.25$pp (4.57\%). Four registered negatives -bracket the ladder: multi-token-prediction as a training auxiliary +bracket the ladder: multi-token-prediction~\cite{mtp} as a training auxiliary scored 5.09\% ($+0.26$pp vs control, with a disclosed preemption confound); swapping news-domain training units for classical Tashkeela at constant budget scored 5.81\% ($+0.98$pp); adding 18k classical units on top at matched epochs scored 4.82\%---flat against the 3-epoch control and 0.25pp behind news-only training at the same epochs, the add canceling the epoch gain entirely; and -on-policy generalized distillation (reverse-KL on student-sampled +on-policy generalized distillation~\cite{gkd} (reverse-KL on student-sampled sequences scored by the frozen teacher) scored 6.00\% ($+1.18$pp, the worst rung measured)---the domain-coverage hypothesis failing in every direction it was tested. Together with the RL negatives, @@ -260,8 +260,8 @@ \subsection{RL at SFT convergence: three negative results} G2P the best checkpoint \emph{lost} 2 points on SentenceBench homographs (77.34$\to$75.37\%), with dev reward degrading monotonically past step 200. - \item \textbf{GTPO-GRPO} (entropy-weighted per-token credit, - arXiv:2508.04349; graded alignment-based reward): dev curve + \item \textbf{GTPO-GRPO} (entropy-weighted per-token + credit~\cite{gtpo}; graded alignment-based reward): dev curve \emph{exactly} flat (5.9692\% at steps 0/100/150); the final benchmark equals the teacher within protocol noise. \end{itemize} @@ -318,6 +318,11 @@ \section{Reproducibility} \begin{thebibliography}{9} \bibitem{sadeed} Sadeed: Arabic Diacritization. arXiv:2504.21635, 2025. +\bibitem{byt5} Xue et al. ByT5: Towards a token-free future with pre-trained byte-to-byte models. TACL, 2022. arXiv:2105.13626. +\bibitem{muon} Liu et al. Muon is Scalable for LLM Training. arXiv:2502.16982, 2025. +\bibitem{mtp} Gloeckle et al. Better \& Faster Large Language Models via Multi-token Prediction. arXiv:2404.19737, 2024. +\bibitem{gkd} Agarwal et al. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. ICLR, 2024. arXiv:2306.13649. +\bibitem{gtpo} Tan et al. GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy. arXiv:2508.04349, 2025. \end{thebibliography} \end{document} diff --git a/docs/paper-hebrew/main.pdf b/docs/paper-hebrew/main.pdf index bc2f91b..aa28f6e 100644 Binary files a/docs/paper-hebrew/main.pdf and b/docs/paper-hebrew/main.pdf differ diff --git a/docs/paper-hebrew/main.tex b/docs/paper-hebrew/main.tex index 423b85c..65933b2 100644 --- a/docs/paper-hebrew/main.tex +++ b/docs/paper-hebrew/main.tex @@ -22,7 +22,7 @@ ACL-2020 test corpora the same model is strong across modern, poetry, and rabbinic line-level registers (0.52/0.24/0.45 DER), localizing the weak surface specifically to paragraph-level Biblical text---register -coverage, not a general modern-text gap; we note D-Nikud, the recent +coverage, not a general modern-text gap; we note D-Nikud~\cite{dnikud}, the recent open TavBERT+Bi-LSTM system, publishes no numbers on these corpora. Error analysis shows the vowel-pointing accuracy of our model at aligned positions is 97.1\%, @@ -204,8 +204,14 @@ \section{Reproducibility} \begin{thebibliography}{9} \bibitem{dictabert} DictaBERT-large-char-menaked. Dicta, 2023--2024. -\bibitem{nakdimon} Elazar et al. Nakdimon. 2020. -\bibitem{byt5} Xue et al. ByT5: Towards a Token-Free Future. 2022. +\bibitem{nakdimon} Elazar Gershuni and Yuval Pinter. Restoring Hebrew + Diacritics Without a Dictionary (Nakdimon). Findings of NAACL, 2022. + arXiv:2105.05209. +\bibitem{byt5} Xue et al. ByT5: Towards a token-free future with + pre-trained byte-to-byte models. TACL, 2022. arXiv:2105.13626. +\bibitem{dnikud} Adi Rosenthal and Nadav Shaked. D-Nikud: Enhancing + Hebrew Diacritization with LSTM and Pretrained Models. arXiv:2402.00075, + 2024. \end{thebibliography} \end{document} diff --git a/docs/paper-umbrella/main.pdf b/docs/paper-umbrella/main.pdf index 5d3c5a9..ec024c8 100644 Binary files a/docs/paper-umbrella/main.pdf and b/docs/paper-umbrella/main.pdf differ diff --git a/docs/paper-umbrella/main.tex b/docs/paper-umbrella/main.tex index 4cef7b7..6a4dcc3 100644 --- a/docs/paper-umbrella/main.tex +++ b/docs/paper-umbrella/main.tex @@ -58,7 +58,7 @@ \section{Why phonology for transliteration?} vocalization that makes interscript.org's maps deterministic. \section{System and results} -Common framework: ByT5 seq2seq (byte-level; tokenizer = UTF-8 bytes, so +Common framework: ByT5~\cite{byt5} seq2seq (byte-level; tokenizer = UTF-8 bytes, so TS inference needs no vocabulary assets) except Arabic, where a compact 30M char-level encoder won on cost. @@ -69,8 +69,8 @@ \section{System and results} \toprule Language & Task & Result & Published reference \\ \midrule -Arabic & diacritization & \textbf{2.29/1.33\% DER} & Sadeed 7.29/5.26\%, Gemini 3.19/2.38\%, GLM-5.3-Flash 8.80 (same benchmark+evaluator) \\ -Hebrew & diacritization & \textbf{16.43\% DER} (Biblical) & DictaBERT 35.6\% (same test) \\ +Arabic & diacritization & \textbf{2.29/1.33\% DER} & Sadeed~\cite{sadeed} 7.29/5.26\%, Gemini 3.19/2.38\%, GLM-5.3-Flash 8.80 (same benchmark+evaluator) \\ +Hebrew & diacritization & \textbf{16.43\% DER} (Biblical) & DictaBERT~\cite{dictabert} 35.6\% (same test) \\ Thai & G2P & \textbf{2.32\% PER} & baseline 6.37\% (same test) \\ Persian & G2P/HA & \textbf{89.5\%} / 77.3\% (SB) & Homo-GE2PE 76.9\% \\ Persian & diacritization & \textbf{0.52\% CER} & (none existed) \\ @@ -151,4 +151,11 @@ \section{Reproducibility} evaluation harnesses, \texttt{docs/RESULTS.md} ground truth, and the five companion papers in \texttt{docs/paper-*/}. +\begin{thebibliography}{9} +\bibitem{sadeed} Sadeed: Arabic Diacritization. arXiv:2504.21635, 2025. +\bibitem{byt5} Xue et al. ByT5: Towards a token-free future with + pre-trained byte-to-byte models. TACL, 2022. arXiv:2105.13626. +\bibitem{dictabert} DictaBERT-large-char-menaked. Dicta, 2023--2024. +\end{thebibliography} + \end{document}