Aligned splits, take 2. - #9409
mcourteaux wants to merge 55 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9409 +/- ##
==========================================
- Coverage 70.12% 69.94% -0.18%
==========================================
Files 261 261
Lines 79938 80138 +200
Branches 19478 19532 +54
==========================================
Hits 56053 56053
- Misses 18056 18129 +73
- Partials 5829 5956 +127 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| mask = select(base == old_base, likely(const_true()), mask); | ||
| Expr mask; | ||
| if (split.align.defined()) { | ||
| // Because base is anchored to align instead of old_min, the |
There was a problem hiding this comment.
I think we have a nested tail strategy tail that tries lots of things in combination. It would be good to add aligned splits to it to get more coverage of this.
There was a problem hiding this comment.
I have two tests added: split_aligned_nested and rfactor_split_aligned_nested which do this. I'm a bit hesitant to conflate the existing nested_tail_strategies with another axis of tests.
e7f5f89 to
9cbb7c1
Compare
9cbb7c1 to
49501ce
Compare
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Gemini Pro 3.1 <gemini@aistudio.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
… those blend operations in case of aligned splits. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Fix old copy-paste bug in simplifier rules. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rename split_aligned_2d_6x6.cpp to split_aligned_2d_3x3.cpp and shrink the pattern to 3x3, which reproduces the surviving mux with a much smaller amount of IR to read. Also fix the test itself: realize the 3-D output with a 3-D shape, check all three channels, sweep all nine (offset_x, offset_y) alignments, and include c in the reorder so it stays innermost. With c left outermost it was unrolled around the xo/yo nest, triplicating the loop nest and recomputing R/G/B once per channel. The test currently fails at the mux count (27 = 9 tile positions x 3 channels); the runtime results are correct for every alignment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A loop of eight whose first and last iterations are special and whose interior is periodic with period two. Unrolling the interior by two folds the % away, but only if the unrolled pairs line up with the periodicity, which means the tiles have to start where the interior does. An aligned split says exactly that, and partitioning then peels one iteration at each end rather than two, leaving a steady-state loop of three rather than two. Checks the extent of the remaining loop, that the modulo folded away, and the values. Dropping the alignment from the split fails the extent check, so the test is measuring the thing it claims to.
|
|
||
| rewrite(x * c0 + y * c1, (x + y * fold(c1 / c0)) * c0, c1 % c0 == 0) || | ||
| rewrite(x * c0 + y * c1, (x * fold(c0 / c1) + y) * c1, c0 % c1 == 0) || | ||
| rewrite(x * c0 + (y * c1 + z), (x * fold(c0 / c1) + y) * c1 + z, c0 % c1 == 0) || |
There was a problem hiding this comment.
The variant of this where c1 % c0 == 0 should be added too, for uniformity with the pair of rules above
There was a problem hiding this comment.
Adding this trips up nested_tail_strategies. It rewrites some bounds Expr that used to cancel into a form that no longer cancels. Precisely the opposite of what the new rule here tries to work around.
The rule we added here was very ad-hoc to make the +offset and up in the back to let the simplifier peel. As the c1 % c0 == 0 counterpart rule breaks things for now, and we are pushing back deciding what to do with this more general simplifier problem, I'd argue to just leave this rule alone without the counterpart you suggest.
I can add a comment about it though.
Note that none of the 3 rules in this block adhere to the reduction order, according to the verifier, but do according to your "constant becomes smaller" rule. This is probably something that needs to be addressed properly in the verifier, or these rules should be kicked out.
| rewrite((x / w) * w + (z + x % w), select(w == 0, 0, x) + z) || | ||
| rewrite(x / 2 + x % 2, (x + 1) / 2) || | ||
|
|
||
| rewrite((0 - (x % 2)) / 2 * 2 + (x % 2), 0 - (x % 2)) || |
There was a problem hiding this comment.
I think this should simplify by a different chain. First:
(0 - x % 2) / 2 is just (0 - x % 2), and then we have the pattern (0 - y) * 2 + y which I think will simplify already.
There was a problem hiding this comment.
Neither of these rules are present in the respective files. Will add those.
There was a problem hiding this comment.
Adding in the first rules brings the simplifier to: ((x % 2)*-1) which seems like a suboptimal endpoint, so I added rewrite(x * -1, 0 - x) in Simplify_Mul to have it land on (0 - (x % 2)). Does that sound right?
| rewrite(x + ((c0 - x) / c1) * c1, c0 - ((c0 - x) % c1), c1 > 0) || | ||
| rewrite(x + ((c0 - x) / c1 + y) * c1, y * c1 - ((c0 - x) % c1) + c0, c1 > 0) || | ||
| rewrite(x + (y + (c0 - x) / c1) * c1, y * c1 - ((c0 - x) % c1) + c0, c1 > 0) || | ||
| rewrite(((0 - x) / c0) + ((x % c0 + c1) / c0), fold(c1 / c0) - (x / c0), c0 > 0 && (c1 + 1) % c0 == 0) || |
There was a problem hiding this comment.
This seems like a hyper-specific rule. What's it for? Is this a pattern produced by the new split logic?
There was a problem hiding this comment.
Yes, it's hyper specific indeed. Disabling breaks one of the tests (correctness_aligned_split_mux_phase). Verified by z3 and the verifier.
|
Many comments on the simplifier rules. Were they all verified? |
|
I'll delete all the simplifier rules, and see what breaks, because I don't know anymore by now. |
SolveExpression::visit(Add) had no case for a zero operand, so a cancelled or negated term left a stray `+ 0` in the solved expression. This surfaced once `x * -1` started canonicalizing to `0 - x`: the Div visitor's `(f(x)*a)/b -> f(x)*(a/b)` rewrite produces `0 - x`, the Sub visitor turns that into `negate(x) + 0`, and nothing folded it away. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AQ1NNAmM1ki6K5L4DC9ERu
… found by Claude, verified by z3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andrew Adams <andrew.b.adams@gmail.com>
…n/max The two factoring rules violated the reduction order and, combined with the Param reassociation and the Let peeler, dissolved tile-base lets in nested splits, doubling allocations in nested_tail_strategies. The Anderson2021 train_cost_model unroll relied on them only to canonicalize two sums written in opposite order; eight z3-verified Simplify_Sub rules now cancel such a subtracted sum directly. Simplify test expectations updated to the unfactored form. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B2B5QccKbpiegADpAFgSBM
A variable without a let binding of its own, such as a loop variable or a Param, scaled by a constant can be pushed into the uses like a bare variable: substituting it can never pull in another let value. Lets like tid*8 + bid*64 then dissolve, so later passes see the loop indices directly instead of an opaque name. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B2B5QccKbpiegADpAFgSBM
| rewrite((x * c0 + c1) * (x * c2 + c3), x * (x * fold(c0 * c2) + fold(c0 * c3 + c1 * c2)) + fold(c1 * c3)))) || | ||
| rewrite((x + c0) * c1, x * c1 + fold(c0 * c1), !overflows(c0 * c1)) || | ||
| rewrite((c0 - x) * c1, x * fold(-c1) + fold(c0 * c1), !overflows(c0 * c1)) || | ||
| rewrite(x * -1, 0 - x) || |
There was a problem hiding this comment.
Until we fix matching in general, this needs to be
| rewrite(x * -1, 0 - x) || | |
| rewrite(x * c0, 0 - x, c0 + 1 == 0) || |
|
Claude Opus 5.5 found a bug here while mechanizing the splits system I'm working on in Lean. // Minimal reproducer: aligned_split + TailStrategy::ShiftInwardsAndBlend on an
// update definition skips required points when the required extent n is less
// than the split factor k and the anchor is not congruent to the min (mod k).
//
// Expected: every element of out is 1.
// Actual (k = 8, anchor = 0, out over [5, 10]): 1 0 0 1 1 1
#include "Halide.h"
#include <cstdio>
using namespace Halide;
int main(int argc, char **argv) {
Var x("x"), xo("xo"), xi("xi");
Func f("f"), out("out");
f(x) = 0;
f(x) = f(x) + 1;
out(x) = f(x);
f.compute_root();
f.update().aligned_split(x, xo, xi, 8, 0, TailStrategy::ShiftInwardsAndBlend);
Buffer<int> result(6);
result.set_min(5); // required region of f is [5, 10]: straddles the anchor-0 grid line at 8
out.realize(result);
int errors = 0;
for (int i = result.min(0); i < result.min(0) + result.extent(0); i++) {
printf("out(%d) = %d\n", i, result(i));
if (result(i) != 1) {
errors++;
}
}
if (errors) {
printf("FAIL: %d element(s) were never updated\n", errors);
return 1;
}
printf("Success!\n");
return 0;
}Its explanation: Why it fails. The required region is [5, 10], so n = 6 < k = 8, and it crosses the anchor-0 grid line at 8. • Both tiles, for cells [0,7] and [8,15], get clamped to the same base, 3. Each one covers [3,10]. The masks come from main , where at most one tile is ever clamped per side. With anchoring and n < k, the two clamped tiles are the same tile. When it fails. Exactly when n < k and a ≢ m (mod k). The sweep found 1862 bad configurations out of 6080, and that is exactly this set. The unaligned split and the other aligned strategies (RoundUpAndBlend, GuardWithIf, RoundUp) had no failures. Suggested fix. Mask each tile to its own anchored cell: keep xi when 0 ≤ xi + (base − (a + j·k)) < k . Here that keeps 3–7 in tile 0 and 8–10 in tile 1. |
When the required extent is smaller than the split factor, the required region can straddle a line of the anchored grid. Two tiles then cover it, and low_bound > high_bound, so a tile can be moved by both the Max and the Min clamp. The old mask looked at only whichever bound the unclamped base crossed, so both tiles could mask out the same middle elements. For example, with factor 8, anchor 0 and region [5, 10], f(6) and f(7) were never updated. Mask each tile to its own grid cell instead: keep inner iff 0 <= inner + (clamped_base - old_base) < factor. This equals the old mask whenever extent >= factor, so steady-state codegen is unchanged. Adds correctness_aligned_split_blend_small_extent, which sweeps every anchor phase and every region with min in [-9, 9] and extent in [1, 19], for ShiftInwardsAndBlend and RoundUpAndBlend, scalar and vectorized. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… a constant A producer computed at the gpu_blocks of a gpu_tile'd consumer, inside an outer GuardWithIf split. The fused thread block size must fold to the tile size instead of depending on the image height. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUvLgw5L5hwGi15HVVfH8h
… c1) - x min(x + c1, c2) canonicalizes to min(x, c2 - c1) + c1, which hid x from the existing clamped-term Sub rules. When SimplifyCorrelatedDifferences solved the thread extent of a producer computed at a gpu_block (tile base b*16 + yso*1024) for the outer loop var, it produced min(E, min(b*16, 1009) + 15) - b*16, whose bound over b is loose. The fused GPU block size then depended on the image height instead of folding to the tile size. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FUvLgw5L5hwGi15HVVfH8h
…alide/Halide into mcourteaux/aligned-split-clean
Inner loops go from 0 to factor, to help with constant bounds analysis.
2D tiled test: compute_at test overwrites the compute and storage bounds by just passing those in the schedule.
It's now possible to align the first iteration of the inner loop, like so:
Replaces #9371
Breaking changes
Checklist