Conversation
ericxiesg
marked this pull request as ready for review
September 11, 2026 03:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Gets FIDESlib building and running on a V100 (sm_70) with CUDA 12.x. Two parts:
FIDESLIB_ARCHisn't set, it reads the GPUs fromnvidia-smiand the toolkit version fromnvcc --version, then builds for the GPUs that nvcc actually supports. With no GPU visible, you get the old fat list clipped to what nvcc accepts.-DFIDESLIB_ARCH=70-real(or any list) still overrides all of it.Why
The default arch list starts at
80-real, so a V100 got no kernel image at all. No single hardcoded list works everywhere: CUDA 13 dropped sm_70, and toolkits older than 12.8 don't know sm_100/sm_120.Once it did run, these came up:
EvalMultByI, then an illegal memory access, thencudaMemcpy 'invalid argument'(N = 2^16, dnum = 3)L + K>MAXP, so the special primes get written past the end of the[MAXP]constant tables. Nothing checked it.SetupConstantsnow throws with the actual counts and what to change (depth,dnumorMAXP)LTdotProductPtBatchreading 35 limbs from a plaintext that holds 34 (bootstrap, FIXEDMANUAL)EvalCoeffsToSlotsdrops the ciphertext to each layer's diagonal level before the product, which is what OpenFHE'sAdjustLevelsAndDepthdoes implicitlyGPUfreeseveral calls latermultMonomialcan index pastlimbon a pooled polynomial that wasn't grown to its levelLTdotProductPtBatchalso checks every operand's limb count against the launch size now, so a level mismatch fails with a readable message instead of a garbage device pointer.API fixes in
api/CryptoContext.cpp:EvalSub(scalar, ct): on GPU this returnedct - scalar. Dropped the extramultScalar(-1.0).EvalMult(ct, pt)/EvalMultInPlace(ct, pt), CPU fallback: theseany_casttoConstPlaintext, butpt->cpualways holds aPlaintext, so they threwbad_any_cast.~CryptoContextImpl: it called the no-argClearEvalMultKeys()/ClearEvalAutomorphismKeys(), which wipe OpenFHE's global key maps, i.e. every other live context's keys too. Removed. OpenFHE's ownCryptoContextImpldoesn't do this either.Also,
KeySwitchingKeyheld aContext&that dangled onceLoadContextmoved its localContextintostd::any. It's now astd::weak_ptr<ContextData>behind acontext()accessor. It's deliberately not ashared_ptr: keys live insideContextData, so owning it would make a cycle and no context would ever be freed.Why it is safe
LTdotProductPtBatchcheck skipsnullptrdiagonals, which the kernel skips as well. An earlier version of the check didn't, and that broke a working bootstrap.EvalCoeffsToSlotsonly ever drops the ciphertext, and only when it sits above that layer's diagonals. When the levels already match, it's a no-op.FIDESLIB_ARCHisn't set. Explicit values behave exactly as before.Testing
The bugs above were hit and fixed on a GV100 (CUDA 12.x, sm_70). A full
fideslib-testrun on the V100 is in progress; I'll post the results here.Notes
-real, no PTX). That's great locally, but not what you want for a package or CI build, so passFIDESLIB_ARCHexplicitly there. Happy to make auto-detection opt-in instead if you'd prefer that.EvalLinearTransform) doesn't get the alignment yet. If the same one-limb gap exists there, it now fails with the limb-count error instead of reading out of bounds.AddBootstrapPlaintextsprints one diagnostic line to stderr per bootstrap setup. It was handy while chasing the level gap, and I can put it behind a flag..gitignore: added__pycache__/,*.pycandbuild-*/.