fix: call .eval() on tokenizer and model in KronosPredictor - #382
Open
hexonal wants to merge 1 commit into
Open
Conversation
`KronosPredictor.__init__` 把 tokenizer 与 model 搬上设备后没有切到 eval 模式, 而 Kronos-small 的配置里 `attn_dropout_p=0.1`。后果分两种,都不好: - Apple Silicon(MPS)上直接崩。`auto_regressive_inference` 整个包在 `torch.no_grad()` 里,而 MPS 的 SDPA 不支持「dropout_p>0 且 grad 关闭」这个组合。 - CPU/CUDA 上不报错,但**带着 dropout 做推理** —— 输出带随机噪声、同一输入两次 调用结果不同,而使用者完全看不出来。 改法是构造时就 `.eval()`。这不改变任何训练路径:`finetune/` 下的训练脚本自己 调 `.train()`,而 KronosPredictor 从设计上就只用于推理。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
KronosPredictor never switches the modules out of training mode, so inference runs with dropout active.
Impact
Kronos-small ships
attn_dropout_p = 0.1. BecauseKronosPredictor.__init__only does.to(device), the modules stay intrain()mode for the whole lifetime of the predictor:F.scaled_dot_product_attentionwith a non-zerodropout_pis not implemented for the MPS backend, so anypredict()on Apple Silicon raises rather than returning a forecast.sample_countsampling noise.Measured on this fork: with
sample_count=5on 60m US equity bars, adding.eval()cut MAE by roughly a fifth on an otherwise unchanged pipeline.Fix
.eval()is the conventional place for this — the predictor is inference-only, and nothing in the class ever calls.train(). Callers who genuinely want stochastic modules can still flip them back after construction.Two lines, no behavioural change for anyone who was already calling
.eval()themselves.