[WIP] Implicit derivatives for CTMRG by leburgel · Pull Request #413 · QuantumKitHub/PEPSKit.jl

leburgel · 2026-08-06T14:57:42Z

Adds implicit gradients for CTMRG contractions.

Reference: https://arxiv.org/abs/2607.15030

codecov · 2026-08-06T21:50:20Z

Codecov Report

❌ Patch coverage is 89.31116% with 45 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...hms/contractions/ctmrg/characteristic_equations.jl	87.17%	35 Missing ⚠️
...lgorithms/optimization/implicit_differentiation.jl	92.56%	9 Missing ⚠️
src/algorithms/ctmrg/sequential.jl	96.15%	1 Missing ⚠️

Files with missing lines	Coverage Δ
src/Defaults.jl	`85.71% <ø> (ø)`
src/algorithms/ctmrg/c4v.jl	`83.48% <100.00%> (-0.45%)`	⬇️
src/algorithms/optimization/peps_optimization.jl	`94.73% <ø> (ø)`
src/algorithms/ctmrg/sequential.jl	`97.70% <96.15%> (-0.72%)`	⬇️
...lgorithms/optimization/implicit_differentiation.jl	`90.59% <92.56%> (ø)`
...hms/contractions/ctmrg/characteristic_equations.jl	`87.17% <87.17%> (ø)`

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2026-08-07T08:30:46Z

After the build completes, the updated documentation will be available here

…icitGradient` is limited to `SimultaneousCTMRG{<:HalfInfiniteProjector}`

lkdvos · 2026-08-11T09:36:44Z

+
+# north edge renormalization
+function contract_E´(Ud::RightProjector{S, 3}, EnVd::LeftProjector{S, 5}) where {S}
+    @tensor E´[-1 -2 -3; -4] := Ud[-1; 1 2 3] * EnVd[1 2 3 -2 -3; -4]


This looks slightly fishy for fermions, but I might also be wrong. For the CTMRG contractions, when we use U' and Vd', don't we use * instead of contractions?

Yes, this definitely broken for fermions. But I think currently everything about the C4v case is broken for fermions. Currently we're not allowing anyone to run a leading_boundary for any Fermionic() braiding style by throwing an explicit argument error. So it's not possible to get here in the first place when optimizing a fermionic network.

I can put another check on the braiding style at the start of the generate_symmetric_characteristic_equation function if that's better.

lkdvos · 2026-08-11T09:39:12Z

+        C´ = Ud * EwCEnVd
+        C´ = _project_hermitian(C´) # project output
+        λ_C = dot(C, C´)
+        F1 = C´ / λ_C - C


Suggested change

F1 = C´ / λ_C - C

F1 = add!!(C´, C, -1, inv(λ_C))

I'm not entirely sure, but we might have rules set up for this such that it remains AD-able

same suggestion on the other parts in this function

Well, it seems this tries to differentiate straight through LinearAlgebra.BLAS.axpby! and then crashes. I can see what I think are VectorInterface rules in TensorKitMooncakeExt, but it seems these just aren't there for ChainRules.

lkdvos · 2026-08-11T09:52:08Z

+            copyto!(block(F, c), _fourthroot_pullback(b))
+        end
+        return NoTangent(), P_t(_elementwise_mult(ΔC, F))
+    end
+    return C, fourthroot_pullback
+end
+function _fourthroot_pullback(C::AbstractMatrix)
+    Fdata = similar(C)
+    for i in axes(Fdata, 1), j in axes(Fdata, 2)
+        # Taking the diagonal only is okay, when dA is diagonal anyway: Fdata[i, i] = 1 / (4 * conj(C[i, i]^3))
+        # However, for Q-deformed CTMRG, we need the full version:
+        Fdata[i, j] = 1 / conj(C[i, i]^3 + C[i, i] * C[j, j]^2 + C[i, i]^2 * C[j, j] + C[j, j]^3)


I think here it might be worth it to be slightly more careful about the memory, since we're now allocating a block in _fourthroot_pullback to immediately copy into the destination of F, which is then copied again in elementwise_mult, and I guess this is not really that heavy of toll it still kind of adds up.

It also feels like there is probably a broadcasting way of writing this, which makes it GPU compatible as well as removing the intermediates, something like this might work? (untested)

Cd = diagview(C) Ct = transpose(Cd) @. F = ΔC / conj(Cd^3 + Cd * Ct^2 + Cd^2 * Ct + Ct^3)

lkdvos · 2026-08-11T09:55:12Z

+    env_fixed = fix_phases(env_conv, signs, corner_phases, edge_phases)
+
+    # NOTE: explicitly keeping corner non-diagonal and (possibly) complex, for use in the backwards pass
+    C, E = only(env_fixed.corners[NORTHWEST, :, :]), only(env_fixed.edges[NORTH, :, :])


Is there a reason for doing the slicing with : and then immediately calling only? would env.corners[NORTHWEST, 1, 1] not give the same result?

I just liked the fact that it would error if you accidentally put in an environment with a non-trivial unit cell, but I'm not sure if it's even possible that this wouldn't be caught before ending up here.

lkdvos · 2026-08-11T09:55:57Z

+        _, ΔC, ΔE = c4v_env_vjp(Δenv)
+
+        # apply Hermitian projectors to edge and corner cotangents
+        ΔE = _project_hermitian(ΔE)


I think the hermitian projectors in the versions of TensorKit we are using now are correctly differentiable

I can use the TensorKit projector for the corners, but not for the edges. For the latter it's a very shady definition of "Hermitian" which assumes additional trivial spaceflips and acts on a tensor map that is very much not square, but there's no real way around that I think.

pbrehmer

Looks really good to me! I would personally like to give SequentialCTMRG with implicit differentiation a try but we can of course also postpone this to a later PR.

pbrehmer · 2026-08-11T10:13:47Z

+
+function _check_algorithm_combination(::SequentialCTMRG, ::ImplicitGradient)
+    msg = "The `:ImplicitGradient` algorithm is not yet compatible with the `SequentialCTMRG` algorithm; select \
+          SimultaneousCTMRG instead to use :ImplicitGradient"


As you suggested, I think we could just try to return the SVD tensors in the ctmrg_iteration of SequentialCTMRG, right? I'm still not super sure if this will actually lead to an element-wise converged environment but I think it should.

Okay I gave this a try and didn't manage to get it to work with ImplicitGradient. The characteristic equations are apparently not fulfilled which I somewhat suspected. On the other hand, SequentialCTMRG does seem to be compatible with FixedPointGradient. I guess now the question is if we still want SequentialCTMRG to return the SVD tensors and perhaps try to circumvent the problems by running one iteration of SimultaneousCTMRG.

Co-authored-by: Lukas Devos <ldevos98@gmail.com>

Co-authored-by: Paul Brehmer <paul.brehmer@univie.ac.at>

…dPointDifferentiation`

… lb/implicit

Add implicit gradient for C4vCTMRG

ce0fe93

leburgel marked this pull request as draft August 6, 2026 14:57

Remove unused partial contractions, add docstring

3c5e81f

leburgel added the documentation Improvements or additions to documentation label Aug 7, 2026

leburgel and others added 5 commits August 7, 2026 15:39

Add implicit differentiation of asymmetric CTMRG

9740499

Fix typo

4f30294

Resolve ambiguities

8ca3250

Fixes

0d1cc8e

FixedPointGradient works fine with SequentialCTMRG now, but `Impl…

758f633

…icitGradient` is limited to `SimultaneousCTMRG{<:HalfInfiniteProjector}`

leburgel marked this pull request as ready for review August 10, 2026 14:26

leburgel requested review from lkdvos and pbrehmer August 10, 2026 14:27

lkdvos reviewed Aug 11, 2026

View reviewed changes

pbrehmer reviewed Aug 11, 2026

View reviewed changes

leburgel and others added 14 commits August 11, 2026 14:58

Update src/algorithms/contractions/ctmrg/characteristic_equations.jl

5cd582f

Co-authored-by: Lukas Devos <ldevos98@gmail.com>

Update src/algorithms/contractions/ctmrg/characteristic_equations.jl

ec009f9

Co-authored-by: Lukas Devos <ldevos98@gmail.com>

Update src/algorithms/optimization/implicit_differentiation.jl

ad25b20

Co-authored-by: Lukas Devos <ldevos98@gmail.com>

Update src/algorithms/optimization/implicit_differentiation.jl

94b8edb

Co-authored-by: Lukas Devos <ldevos98@gmail.com>

Update src/algorithms/contractions/ctmrg/characteristic_equations.jl

9c9e086

Co-authored-by: Paul Brehmer <paul.brehmer@univie.ac.at>

Add reference

d00622b

Remove unused routines, use builtin projection, and more add!!

e9ee841

Specify

570d6c0

Revert add!!s

86a89eb

Docstring additions

5b4bb5c

Make SequentialCTMRG return SVD tensors

f9559de

Fix typo

0702ff6

Make SequentialCTMRG differentiable again and compatible with `Fixe…

22f875a

…dPointDifferentiation`

Merge branch 'lb/implicit' of github.com:quantumghent/PEPSKit.jl into…

b691c67

… lb/implicit

Fix unequal domain codomain initialization

fbe4240

Uh oh!

Conversation

leburgel commented Aug 6, 2026

Uh oh!

codecov Bot commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pbrehmer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Aug 6, 2026 •

edited

Loading