Skip to content

Thread tanh logit softcapping through FlashAttention (FA2 and FA3) - #3391

Open
nvegesna-netizen wants to merge 37 commits into
NVIDIA:mainfrom
nvegesna-netizen:nvegesna/gemma2-softcap-core
Open

Thread tanh logit softcapping through FlashAttention (FA2 and FA3)#3391
nvegesna-netizen wants to merge 37 commits into
NVIDIA:mainfrom
nvegesna-netizen:nvegesna/gemma2-softcap-core

fix(pytorch): reject a negative softcap instead of diverging silently

f0cd672
Select commit
Loading
Failed to load commit list.
DCO / DCO succeeded Sep 7, 2026 in 1s

DCO

All commits are signed off!

View more details on DCO