[tanmay442]

Note

State of this PR. One file, src/first_last.rs (+130/-48). It replaces the
per-batch sweep of the per-group scoreboard with the list of groups the batch
touched, the idiom the pre-ordered path has used since #25050. The invariant
is that the scoreboard is all-false between batches, and both paths clear
their own bits.

No benchmark-runner measurement of this implementation exists yet. The two
benchmark reports in the thread were run on 522db22, an earlier
BitIndexIterator version that was force-pushed away on 21 Oct. Rows with an
empty baseline column are unpaired, not wins. The new cases need #26554 in
the merge base; the eight rows that came out slower do not, because both
sides of this pull request already carry the same benchmark file, and a run
of just those eight is what would settle them.

Which issue does this PR close?

• N/A

Rationale for this change

This is a performance change. The rows a query returns, the error messages, the
log lines and the row order are the same.

first_value(x ORDER BY y) and last_value(x ORDER BY y) get slower as the
number of groups in the query grows. The grouped accumulator keeps a scoreboard
with one bit per group to record which group already has a winning row in the
current batch. At the end of the batch it swept the whole scoreboard to find
the groups that had a winner, so the work per batch grew with the number of
groups in the query rather than with the number of groups the batch touched.

The file already keeps the list of groups a batch touched, for the pre-ordered
fast path added in #25050. This change uses that list in the tournament path as
well.

What changes are included in this PR?

One file, datafusion/functions-aggregate/src/first_last.rs.

get_filtered_extreme_of_each_group used to reset the scoreboard at the start
of every batch and then sweep it to collect the winners. It now records the
groups it touches while it walks the rows, collects the winners from that list,
and clears exactly those bits again. The per-batch work is proportional to the
number of groups the batch touched.

With both paths restoring the all-false invariant between batches, the
extreme_buf_dirty flag that told the pre-ordered path to reset the whole
scoreboard after an interleaved merge_batch is dead, and is removed. The
pre-ordered path now clears its bits before its loop, so the invariant holds
even when a batch fails.

size() now counts touched_groups_buf, which the tournament path populates
and the accumulator keeps.

The benchmark for this change is in #26554, a separate pull request, as the
contributor guide suggests. The benchmark runner pairs benchmarks by name, so
the new cases need #26554 in the merge base before they can be compared. This
pull request will be rebased once #26554 merges.

Why the behaviour does not change

• The old code walked the winner index array in order and kept the groups whose
  scoreboard bit was set. The new code collects the same groups from the list it
  built while walking the rows. The winner row of each group is the same.
• The winners now come back in the order the groups were first touched rather
  than in group order. Both callers (update_batch and merge_batch) read and
  write state per group, so neither observes the order.
• The scoreboard is all-false between batches, as it was for the pre-ordered
  path. resize_states sizes the scoreboard and the winner index array
  together, so a group index is always inside the array.

Notes for the reviewer

• The changed code is one function. Its two call sites only read and write
  state per group, so the per-group work and its order are the whole story.
• The invariant that matters is "the scoreboard is all-false between batches".
  The new test scoreboard_is_cleared_between_batches checks it, and the
  existing pre_ordered_update_after_merge_interleave checks that the two
  paths can interleave on one accumulator.
• The only per-batch cost this adds is one slot per touched group, for the list
  of touched groups. The winners vector itself is not new.

What is the testing strategy for this PR?

cargo test -p datafusion-functions-aggregate --lib passes, 255 tests, and the
full sqllogictest suite passes, 535 of 535 test files.
cargo test -p datafusion --test core_integration passes, 1442 tests.
cargo clippy -p datafusion-functions-aggregate --all-targets -- -D warnings is
clean.

The invariant that the scoreboard is all-false between batches cannot be
reached through SQL, so it is covered by unit tests in the module:

• winners_are_the_groups_the_batch_touched checks that the winners of a batch
  are exactly the groups that batch touched, for both FIRST_VALUE and
  LAST_VALUE, over group counts of 1, 63, 64, 65, 127, 128, 129 and 100,000.
• scoreboard_is_cleared_between_batches checks that the scoreboard is
  all-false after a batch and that a group touched by the next batch is
  treated as new. Removing the clearing makes both assertions fail: the
  all-false check, and then the value check with the group keeping its old
  value.

The existing test pre_ordered_update_after_merge_interleave covers the
interleaving of the two paths on one accumulator.

Measurements of the current implementation

Author-local, on an x86 machine, compared by criterion against each other. The
base is the benchmark commit b15f964 and the change is 3934ca41, which is
the same benchmark file with the accumulator change applied. These are not the
project runner's numbers, and the machine is not the runner's.

The distinct_batches cases come first, because each batch owns a disjoint
range of groups, so every batch writes the state of every winner. That is the
work a query does, and it is what the change has to be judged by.

case	base	change	delta
first_value update_bench_many_groups_distinct_batches groups=1000000, touched=64	50.76 ms	4.26 ms	-91.6%
first_value update_bench_many_groups_distinct_batches groups=1000000, touched=8192	89.49 ms	38.52 ms	-57.0%
last_value update_bench_many_groups_distinct_batches groups=1000000, touched=64	51.45 ms	4.29 ms	-91.7%
last_value update_bench_many_groups_distinct_batches groups=1000000, touched=8192	96.10 ms	38.62 ms	-59.9%

The many_groups cases replay one batch, so after the first iteration the
accumulator already holds that batch's extreme and the state writes do not
happen. They isolate the per-batch bookkeeping and are not a query shape.

case	base	change	delta
first_value update_bench_many_groups groups=1000000, touched=64	50.44 ms	3.44 ms	-93.2%
first_value update_bench_many_groups groups=1000000, touched=8192	78.80 ms	31.50 ms	-60.0%
last_value update_bench_many_groups groups=1000000, touched=64	50.47 ms	3.45 ms	-93.2%
last_value update_bench_many_groups groups=1000000, touched=8192	80.35 ms	32.89 ms	-59.0%

The dense case is the other end, 100,000 groups and a 100,000 row batch, so
every batch touches every group. That is the shape where the touched list is
longest and the added cost is highest, so it is an upper bound on the price.
The change is flat there. The eight rows that came out slower are 1,024 groups
with 64 rows per group, where the touched list is proportionally far shorter.

case	base	change	delta
first_value update_bench_many_groups_dense groups=100000	403.75 ms	402.81 ms	-0.2%, not significant
last_value update_bench_many_groups_dense groups=100000	406.86 ms	400.85 ms	-1.5%, significant, faster

Measurements from the replaced revision (retained for chronology)

Author-local, on 522db22, the earlier BitIndexIterator version. End to end
through the datafusion CLI, four partitions:

query	before	after	delta
16M rows, 250K groups, 64 rows per group	0.209 s	0.198 s	-5.3%
40M rows, 1M groups, one row per group	1.769 s	1.550 s	-12.4%

These are a lower bound for the shape of the effect, not evidence for this
diff.

Open item

The eight nested last_value update_bench rows measured +1.07% to +2.45% in
the full run and +0.84% to +3.37% in the eight-case rerun, both on the
replaced revision. They are not measured on the project runner for the current
head.

They are measurable now, without #26554, because both sides of this pull
request carry the same benchmark file and a run of just those eight rows would
compare the current head against its parent:

run benchmark first_last
baseline:
  ref: f9bc5a3fc60cc7f4319cd2c156cb3d024a8c1b21
changed:
  ref: 0e0a63c2f35b1fee2122cf3df5d0e75f477f1348
env:
  BENCH_FILTER: '^last_value update_bench (list|map|struct)'

A bench-only null run is what attributes them, because the library is then
identical and the benchmark file is the only difference. Run locally on an x86
machine, the eight nested rows moved between -9.5% and -1.0% and the primitive
last_value update_bench rows between +1.5% and +4.4%, which is the opposite
sign on each group to what the runner measured. The sign follows the case and
not the code. The runner's own null run, main against #26554, is the version to
quote.

Are there any user-facing changes?

No. The same groups win, with the same rows, in the same order.

[kumarUjjawal]

run benchmark first_last

env:
BENCH_FILTER: '^(first_value|last_value) (update_bench|merge_bench)'

[adriangbot]

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6307221599-3710-b57mh 6.12.94+ #1 SMP Fri Aug 21 08:00:16 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing perf/first-last-sparse-group-scoreboard (522db22) to e30aff5 (merge-base) diff

Run configuration

run benchmark first_last
env:
  BENCH_FILTER: "^(first_value|last_value) (update_bench|merge_bench)"

Results will be posted here when complete

---

File an issue against this benchmark runner

[adriangbot]

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing perf/first-last-sparse-group-scoreboard (522db22) to e30aff5 (merge-base) diff

Run configuration

run benchmark first_last
env:
  BENCH_FILTER: "^(first_value|last_value) (update_bench|merge_bench)"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                                HEAD                                   perf_first-last-sparse-group-scoreboard
-----                                                                ----                                   ---------------------------------------
first_value merge_bench list<i64>[4] nulls=0%                        1.01     38.1±0.13ms        ? ?/sec    1.00     37.6±0.08ms        ? ?/sec
first_value merge_bench list<i64>[4] nulls=90%                       1.00     38.3±0.12ms        ? ?/sec    1.00     38.1±0.12ms        ? ?/sec
first_value merge_bench list<struct(i64,utf8)>[4] nulls=0%           1.00     40.9±0.10ms        ? ?/sec    1.00     40.9±0.06ms        ? ?/sec
first_value merge_bench list<struct(i64,utf8)>[4] nulls=90%          1.00     41.6±0.12ms        ? ?/sec    1.00     41.5±0.08ms        ? ?/sec
first_value merge_bench map<utf8,i64> nulls=0%                       1.00     40.5±0.09ms        ? ?/sec    1.00     40.5±0.20ms        ? ?/sec
first_value merge_bench map<utf8,i64> nulls=90%                      1.00     41.9±0.08ms        ? ?/sec    1.00     41.7±0.12ms        ? ?/sec
first_value merge_bench nulls=0%, filter=false                       1.00     35.3±0.06ms        ? ?/sec    1.00     35.4±0.06ms        ? ?/sec
first_value merge_bench nulls=0%, filter=true                        1.00     35.4±0.08ms        ? ?/sec    1.00     35.5±0.05ms        ? ?/sec
first_value merge_bench nulls=90%, filter=false                      1.00     37.8±0.07ms        ? ?/sec    1.01     38.1±0.06ms        ? ?/sec
first_value merge_bench nulls=90%, filter=true                       1.00     37.8±0.07ms        ? ?/sec    1.01     38.1±0.09ms        ? ?/sec
first_value merge_bench struct(i64,utf8,f64) nulls=0%                1.00     40.0±0.10ms        ? ?/sec    1.00     40.0±0.12ms        ? ?/sec
first_value merge_bench struct(i64,utf8,f64) nulls=90%               1.00     42.3±0.15ms        ? ?/sec    1.00     42.2±0.09ms        ? ?/sec
first_value update_bench list<i64>[4] nulls=0%                       1.01     32.2±0.12ms        ? ?/sec    1.00     32.0±0.07ms        ? ?/sec
first_value update_bench list<i64>[4] nulls=90%                      1.00     32.6±0.10ms        ? ?/sec    1.00     32.6±0.08ms        ? ?/sec
first_value update_bench list<struct(i64,utf8)>[4] nulls=0%          1.01     35.5±0.11ms        ? ?/sec    1.00     35.3±0.16ms        ? ?/sec
first_value update_bench list<struct(i64,utf8)>[4] nulls=90%         1.00     36.2±0.10ms        ? ?/sec    1.00     36.3±0.10ms        ? ?/sec
first_value update_bench map<utf8,i64> nulls=0%                      1.00     35.3±0.11ms        ? ?/sec    1.00     35.3±0.14ms        ? ?/sec
first_value update_bench map<utf8,i64> nulls=90%                     1.00     36.6±0.10ms        ? ?/sec    1.00     36.6±0.13ms        ? ?/sec
first_value update_bench nulls=0%, filter=false                      1.00     29.5±0.11ms        ? ?/sec    1.01     29.8±0.09ms        ? ?/sec
first_value update_bench nulls=0%, filter=true                       1.04     46.8±0.18ms        ? ?/sec    1.00     45.1±0.12ms        ? ?/sec
first_value update_bench nulls=90%, filter=false                     1.00     33.9±0.06ms        ? ?/sec    1.00     33.8±0.09ms        ? ?/sec
first_value update_bench nulls=90%, filter=true                      1.03     47.1±0.14ms        ? ?/sec    1.00     45.7±0.09ms        ? ?/sec
first_value update_bench struct(i64,utf8,f64) nulls=0%               1.00     34.7±0.09ms        ? ?/sec    1.00     34.7±0.11ms        ? ?/sec
first_value update_bench struct(i64,utf8,f64) nulls=90%              1.00     37.2±0.12ms        ? ?/sec    1.00     37.0±0.08ms        ? ?/sec
first_value update_bench_many_groups groups=1000000, touched=64                                             1.00     24.4±0.42ms        ? ?/sec
first_value update_bench_many_groups groups=1000000, touched=8192                                           1.00     48.7±0.40ms        ? ?/sec
last_value merge_bench list<i64>[4] nulls=0%                         1.01     38.3±0.12ms        ? ?/sec    1.00     37.8±0.06ms        ? ?/sec
last_value merge_bench list<i64>[4] nulls=90%                        1.00     38.7±0.14ms        ? ?/sec    1.00     38.6±0.14ms        ? ?/sec
last_value merge_bench list<struct(i64,utf8)>[4] nulls=0%            1.01     41.2±0.10ms        ? ?/sec    1.00     40.9±0.09ms        ? ?/sec
last_value merge_bench list<struct(i64,utf8)>[4] nulls=90%           1.00     41.9±0.10ms        ? ?/sec    1.00     41.8±0.11ms        ? ?/sec
last_value merge_bench map<utf8,i64> nulls=0%                        1.00     40.8±0.11ms        ? ?/sec    1.00     40.7±0.11ms        ? ?/sec
last_value merge_bench map<utf8,i64> nulls=90%                       1.00     42.3±0.10ms        ? ?/sec    1.00     42.1±0.17ms        ? ?/sec
last_value merge_bench nulls=0%, filter=false                        1.00     35.7±0.07ms        ? ?/sec    1.01     36.0±0.05ms        ? ?/sec
last_value merge_bench nulls=0%, filter=true                         1.00     35.9±0.11ms        ? ?/sec    1.00     35.9±0.05ms        ? ?/sec
last_value merge_bench nulls=90%, filter=false                       1.01     47.2±0.12ms        ? ?/sec    1.00     46.9±0.13ms        ? ?/sec
last_value merge_bench nulls=90%, filter=true                        1.01     47.6±0.24ms        ? ?/sec    1.00     46.9±0.12ms        ? ?/sec
last_value merge_bench struct(i64,utf8,f64) nulls=0%                 1.01     40.5±0.10ms        ? ?/sec    1.00     40.1±0.14ms        ? ?/sec
last_value merge_bench struct(i64,utf8,f64) nulls=90%                1.01     42.7±0.15ms        ? ?/sec    1.00     42.4±0.09ms        ? ?/sec
last_value update_bench list<i64>[4] nulls=0%                        1.00     32.6±0.12ms        ? ?/sec    1.03     33.4±0.07ms        ? ?/sec
last_value update_bench list<i64>[4] nulls=90%                       1.00     33.1±0.16ms        ? ?/sec    1.02     33.7±0.08ms        ? ?/sec
last_value update_bench list<struct(i64,utf8)>[4] nulls=0%           1.00     35.9±0.11ms        ? ?/sec    1.02     36.5±0.06ms        ? ?/sec
last_value update_bench list<struct(i64,utf8)>[4] nulls=90%          1.00     36.6±0.10ms        ? ?/sec    1.02     37.1±0.09ms        ? ?/sec
last_value update_bench map<utf8,i64> nulls=0%                       1.00     35.6±0.12ms        ? ?/sec    1.01     36.0±0.07ms        ? ?/sec
last_value update_bench map<utf8,i64> nulls=90%                      1.00     37.0±0.10ms        ? ?/sec    1.01     37.5±0.08ms        ? ?/sec
last_value update_bench nulls=0%, filter=false                       1.01     30.6±0.10ms        ? ?/sec    1.00     30.2±0.06ms        ? ?/sec
last_value update_bench nulls=0%, filter=true                        1.03     46.9±0.10ms        ? ?/sec    1.00     45.5±0.09ms        ? ?/sec
last_value update_bench nulls=90%, filter=false                      1.00     42.2±0.08ms        ? ?/sec    1.00     42.1±0.08ms        ? ?/sec
last_value update_bench nulls=90%, filter=true                       1.04     54.0±0.25ms        ? ?/sec    1.00     52.0±0.10ms        ? ?/sec
last_value update_bench struct(i64,utf8,f64) nulls=0%                1.00     35.1±0.09ms        ? ?/sec    1.02     35.8±0.07ms        ? ?/sec
last_value update_bench struct(i64,utf8,f64) nulls=90%               1.00     37.4±0.10ms        ? ?/sec    1.01     37.8±0.10ms        ? ?/sec
last_value update_bench_many_groups groups=1000000, touched=64                                              1.00     24.4±0.58ms        ? ?/sec
last_value update_bench_many_groups groups=1000000, touched=8192                                            1.00     48.8±0.42ms        ? ?/sec

Resource Usage

first_last — base (merge-base)

Metric	Value
Wall time	850.2s
Peak memory	64.6 MiB
Avg memory	40.5 MiB
CPU user	744.3s
CPU sys	0.4s
Peak spill	0 B

first_last — branch

Metric	Value
Wall time	910.2s
Peak memory	501.9 MiB
Avg memory	247.3 MiB
CPU user	799.8s
CPU sys	0.7s
Peak spill	0 B

---

File an issue against this benchmark runner

[kumarUjjawal]

run benchmark first_last

env:
BENCH_FILTER: '^last_value update_bench (list|map|struct)'
baseline:
ref: e30aff5
changed:
ref: 522db22

[adriangbot]

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6307737991-3711-kd7gj 6.12.94+ #1 SMP Fri Aug 21 08:00:16 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 522db22 (522db22) to e30aff5 diff

Run configuration

run benchmark first_last
env:
  BENCH_FILTER: "^last_value update_bench (list|map|struct)"
baseline:
  ref: "e30aff54957c9e16912f5c1102282e2af146f90e"
changed:
  ref: "522db22102ee2cb60b922a3532dadb7b38449be9"

Results will be posted here when complete

---

File an issue against this benchmark runner

[adriangbot]

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 522db22 (522db22) to e30aff5 diff

Run configuration

run benchmark first_last
env:
  BENCH_FILTER: "^last_value update_bench (list|map|struct)"
baseline:
  ref: "e30aff54957c9e16912f5c1102282e2af146f90e"
changed:
  ref: "522db22102ee2cb60b922a3532dadb7b38449be9"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                          HEAD                                   perf_first-last-sparse-group-scoreboard
-----                                                          ----                                   ---------------------------------------
last_value update_bench list<i64>[4] nulls=0%                  1.00     32.6±0.09ms        ? ?/sec    1.03     33.7±0.09ms        ? ?/sec
last_value update_bench list<i64>[4] nulls=90%                 1.00     33.2±0.12ms        ? ?/sec    1.03     34.0±0.08ms        ? ?/sec
last_value update_bench list<struct(i64,utf8)>[4] nulls=0%     1.00     35.9±0.10ms        ? ?/sec    1.01     36.2±0.09ms        ? ?/sec
last_value update_bench list<struct(i64,utf8)>[4] nulls=90%    1.00     36.6±0.09ms        ? ?/sec    1.01     37.0±0.07ms        ? ?/sec
last_value update_bench map<utf8,i64> nulls=0%                 1.00     35.5±0.10ms        ? ?/sec    1.02     36.1±0.10ms        ? ?/sec
last_value update_bench map<utf8,i64> nulls=90%                1.00     36.9±0.08ms        ? ?/sec    1.02     37.5±0.10ms        ? ?/sec
last_value update_bench struct(i64,utf8,f64) nulls=0%          1.00     34.6±0.11ms        ? ?/sec    1.02     35.3±0.10ms        ? ?/sec
last_value update_bench struct(i64,utf8,f64) nulls=90%         1.00     37.4±0.16ms        ? ?/sec    1.01     37.9±0.09ms        ? ?/sec

Resource Usage

first_last — base (merge-base)

Metric	Value
Wall time	170.1s
Peak memory	57.8 MiB
Avg memory	32.6 MiB
CPU user	120.2s
CPU sys	0.1s
Peak spill	0 B

first_last — branch

Metric	Value
Wall time	170.0s
Peak memory	60.5 MiB
Avg memory	34.4 MiB
CPU user	121.7s
CPU sys	0.1s
Peak spill	0 B

---

File an issue against this benchmark runner

[kumarUjjawal]

@tanmay442 Thank you for working on this.

The rerun reproduced slowdowns in all eight cases: approximately 0.8–3.4%, compared with 1.1–2.5% in the first run. The largest was list[4] with no nulls: 32.6 → 33.7 ms.

Could you investigate this regression?

[tanmay442]

@kumarUjjawal I rewrote the implementation after your run, so the numbers you measured belong to the previous version.

The change is a complete overhaul of the mechanism, not of the scope. The pull request still touches one file. The tournament path now collects its winners from the list of groups the batch touched.

Could you trigger the benchmark runner on this pull request so i can get feedback ? Please run the whole first_last suite. The eight nested last_value update_bench rows then land in the same table as everything else.

If those eight rows come back slower, could you run the same trigger on #26554 as well? There the library is the same and only the benchmark file differs, which separates the harness from the code.

[kumarUjjawal]

run benchmark first_last

baseline:
ref: f9bc5a3
changed:
ref: 0e0a63c

[adriangbot]

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6320691868-3732-s8d29 6.12.94+ #1 SMP Fri Aug 21 08:00:16 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 0e0a63c (0e0a63c) to f9bc5a3 diff

Run configuration

run benchmark first_last
baseline:
  ref: "f9bc5a3fc60cc7f4319cd2c156cb3d024a8c1b21"
changed:
  ref: "0e0a63c2f35b1fee2122cf3df5d0e75f477f1348"

Results will be posted here when complete

---

File an issue against this benchmark runner

[kumarUjjawal]

@tanmay442 can you please not paste the whole llm response its hard to figure out what's important from the wall of text. See https://datafusion.apache.org/contributor-guide/ai-policy.html

[adriangbot]

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 0e0a63c (0e0a63c) to f9bc5a3 diff

Run configuration

run benchmark first_last
baseline:
  ref: "f9bc5a3fc60cc7f4319cd2c156cb3d024a8c1b21"
changed:
  ref: "0e0a63c2f35b1fee2122cf3df5d0e75f477f1348"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                                         HEAD                                   perf_first-last-sparse-group-scoreboard
-----                                                                         ----                                   ---------------------------------------
first_value coalesce_peers(i64,utf8,f64) coalesced struct (winner changes)    1.00    441.2±5.01ms        ? ?/sec    1.02    449.7±2.47ms        ? ?/sec
first_value coalesce_peers(i64,utf8,f64) coalesced struct (winner stable)     1.00     35.0±0.13ms        ? ?/sec    1.04     36.3±0.11ms        ? ?/sec
first_value coalesce_peers(i64,utf8,f64) separate x3 (winner changes)         1.00     78.0±0.42ms        ? ?/sec    1.08     84.2±0.30ms        ? ?/sec
first_value coalesce_peers(i64,utf8,f64) separate x3 (winner stable)          1.00     90.5±0.31ms        ? ?/sec    1.03     93.3±0.26ms        ? ?/sec
first_value evaluate_bench list<i64>[4] nulls=0%, all                         1.00    507.1±6.13µs        ? ?/sec    1.00    506.3±5.14µs        ? ?/sec
first_value evaluate_bench list<i64>[4] nulls=90%, all                        1.01    558.9±8.39µs        ? ?/sec    1.00    553.2±7.08µs        ? ?/sec
first_value evaluate_bench list<struct(i64,utf8)>[4] nulls=0%, all            1.04   848.3±18.39µs        ? ?/sec    1.00    818.3±6.02µs        ? ?/sec
first_value evaluate_bench list<struct(i64,utf8)>[4] nulls=90%, all           1.05   854.2±28.25µs        ? ?/sec    1.00   815.6±34.03µs        ? ?/sec
first_value evaluate_bench map<utf8,i64> nulls=0%, all                        1.04    696.2±9.70µs        ? ?/sec    1.00    669.9±9.42µs        ? ?/sec
first_value evaluate_bench map<utf8,i64> nulls=90%, all                       1.00   702.8±48.63µs        ? ?/sec    1.05   738.6±37.57µs        ? ?/sec
first_value evaluate_bench nulls=0%, filter=false, all                        1.00     17.4±1.97µs        ? ?/sec    1.19     20.6±3.37µs        ? ?/sec
first_value evaluate_bench nulls=0%, filter=false, first(2)                   1.00     13.1±1.36µs        ? ?/sec    1.07     13.9±1.49µs        ? ?/sec
first_value evaluate_bench nulls=0%, filter=true, all                         1.00     16.9±1.88µs        ? ?/sec    1.10     18.6±3.41µs        ? ?/sec
first_value evaluate_bench nulls=0%, filter=true, first(2)                    1.00     18.2±2.09µs        ? ?/sec    1.02     18.7±2.32µs        ? ?/sec
first_value evaluate_bench nulls=90%, filter=false, all                       1.00     17.3±1.77µs        ? ?/sec    1.00     17.3±2.52µs        ? ?/sec
first_value evaluate_bench nulls=90%, filter=false, first(2)                  1.06     19.1±2.42µs        ? ?/sec    1.00     18.0±1.94µs        ? ?/sec
first_value evaluate_bench nulls=90%, filter=true, all                        1.00     16.7±1.52µs        ? ?/sec    1.10     18.3±3.26µs        ? ?/sec
first_value evaluate_bench nulls=90%, filter=true, first(2)                   1.05     18.9±2.46µs        ? ?/sec    1.00     18.0±2.01µs        ? ?/sec
first_value evaluate_bench struct(i64,utf8,f64) nulls=0%, all                 1.02   730.3±11.55µs        ? ?/sec    1.00    715.3±6.72µs        ? ?/sec
first_value evaluate_bench struct(i64,utf8,f64) nulls=90%, all                1.04  1110.8±15.43µs        ? ?/sec    1.00  1064.7±48.11µs        ? ?/sec
first_value merge_bench list<i64>[4] nulls=0%                                 1.00     37.9±0.16ms        ? ?/sec    1.00     37.8±0.12ms        ? ?/sec
first_value merge_bench list<i64>[4] nulls=90%                                1.00     38.3±0.12ms        ? ?/sec    1.00     38.3±0.17ms        ? ?/sec
first_value merge_bench list<struct(i64,utf8)>[4] nulls=0%                    1.00     40.7±0.12ms        ? ?/sec    1.00     40.9±0.18ms        ? ?/sec
first_value merge_bench list<struct(i64,utf8)>[4] nulls=90%                   1.00     41.5±0.15ms        ? ?/sec    1.01     41.7±0.12ms        ? ?/sec
first_value merge_bench map<utf8,i64> nulls=0%                                1.00     40.3±0.15ms        ? ?/sec    1.01     40.6±0.13ms        ? ?/sec
first_value merge_bench map<utf8,i64> nulls=90%                               1.00     41.7±0.11ms        ? ?/sec    1.01     41.9±0.16ms        ? ?/sec
first_value merge_bench nulls=0%, filter=false                                1.00     35.7±0.09ms        ? ?/sec    1.00     35.6±0.16ms        ? ?/sec
first_value merge_bench nulls=0%, filter=true                                 1.01     35.7±0.07ms        ? ?/sec    1.00     35.4±0.14ms        ? ?/sec
first_value merge_bench nulls=90%, filter=false                               1.00     37.4±0.06ms        ? ?/sec    1.02     38.3±0.05ms        ? ?/sec
first_value merge_bench nulls=90%, filter=true                                1.00     37.4±0.07ms        ? ?/sec    1.03     38.4±0.06ms        ? ?/sec
first_value merge_bench struct(i64,utf8,f64) nulls=0%                         1.00     40.5±0.19ms        ? ?/sec    1.00     40.4±0.14ms        ? ?/sec
first_value merge_bench struct(i64,utf8,f64) nulls=90%                        1.00     42.5±0.15ms        ? ?/sec    1.01     42.7±0.14ms        ? ?/sec
first_value trivial_update_bench nulls=0%, ignore_nulls=false                 1.00      2.6±0.07µs        ? ?/sec    1.00      2.6±0.06µs        ? ?/sec
first_value trivial_update_bench nulls=0%, ignore_nulls=true                  1.00      2.7±0.10µs        ? ?/sec    1.00      2.7±0.06µs        ? ?/sec
first_value trivial_update_bench nulls=90%, ignore_nulls=false                1.02      3.0±0.20µs        ? ?/sec    1.00      2.9±0.13µs        ? ?/sec
first_value trivial_update_bench nulls=90%, ignore_nulls=true                 1.01      3.6±0.03µs        ? ?/sec    1.00      3.6±0.02µs        ? ?/sec
first_value update_bench list<i64>[4] nulls=0%                                1.00     32.2±0.15ms        ? ?/sec    1.04     33.4±0.08ms        ? ?/sec
first_value update_bench list<i64>[4] nulls=90%                               1.00     32.8±0.13ms        ? ?/sec    1.04     34.0±0.12ms        ? ?/sec
first_value update_bench list<struct(i64,utf8)>[4] nulls=0%                   1.00     35.6±0.15ms        ? ?/sec    1.02     36.3±0.15ms        ? ?/sec
first_value update_bench list<struct(i64,utf8)>[4] nulls=90%                  1.00     36.5±0.13ms        ? ?/sec    1.02     37.2±0.12ms        ? ?/sec
first_value update_bench map<utf8,i64> nulls=0%                               1.00     35.4±0.09ms        ? ?/sec    1.02     36.1±0.18ms        ? ?/sec
first_value update_bench map<utf8,i64> nulls=90%                              1.00     36.8±0.12ms        ? ?/sec    1.02     37.5±0.14ms        ? ?/sec
first_value update_bench nulls=0%, filter=false                               1.00     30.3±0.06ms        ? ?/sec    1.01     30.6±0.10ms        ? ?/sec
first_value update_bench nulls=0%, filter=true                                1.05     47.0±0.08ms        ? ?/sec    1.00     44.9±0.14ms        ? ?/sec
first_value update_bench nulls=90%, filter=false                              1.00     34.1±0.06ms        ? ?/sec    1.00     34.0±0.04ms        ? ?/sec
first_value update_bench nulls=90%, filter=true                               1.02     46.6±0.11ms        ? ?/sec    1.00     45.6±0.10ms        ? ?/sec
first_value update_bench struct(i64,utf8,f64) nulls=0%                        1.00     35.1±0.11ms        ? ?/sec    1.03     36.0±0.15ms        ? ?/sec
first_value update_bench struct(i64,utf8,f64) nulls=90%                       1.00     37.7±0.17ms        ? ?/sec    1.01     38.1±0.17ms        ? ?/sec
last_value evaluate_bench nulls=0%, filter=false, all                         1.00     18.1±2.01µs        ? ?/sec    1.02     18.5±3.06µs        ? ?/sec
last_value evaluate_bench nulls=0%, filter=false, first(2)                    1.02     18.9±2.08µs        ? ?/sec    1.00     18.6±2.06µs        ? ?/sec
last_value evaluate_bench nulls=0%, filter=true, all                          1.00     17.3±2.07µs        ? ?/sec    1.00     17.3±2.65µs        ? ?/sec
last_value evaluate_bench nulls=0%, filter=true, first(2)                     1.03     18.8±2.09µs        ? ?/sec    1.00     18.3±1.61µs        ? ?/sec
last_value evaluate_bench nulls=90%, filter=false, all                        1.00     17.7±1.97µs        ? ?/sec    1.01     17.9±2.78µs        ? ?/sec
last_value evaluate_bench nulls=90%, filter=false, first(2)                   1.03     19.1±1.91µs        ? ?/sec    1.00     18.5±1.58µs        ? ?/sec
last_value evaluate_bench nulls=90%, filter=true, all                         1.00     16.9±1.57µs        ? ?/sec    1.04     17.5±2.29µs        ? ?/sec
last_value evaluate_bench nulls=90%, filter=true, first(2)                    1.00     18.7±2.07µs        ? ?/sec    1.01     18.9±2.16µs        ? ?/sec
last_value merge_bench list<i64>[4] nulls=0%                                  1.00     38.3±2.65ms        ? ?/sec    1.03     39.4±0.12ms        ? ?/sec
last_value merge_bench list<i64>[4] nulls=90%                                 1.00     38.3±0.11ms        ? ?/sec    1.04     39.7±0.17ms        ? ?/sec
last_value merge_bench list<struct(i64,utf8)>[4] nulls=0%                     1.00     41.0±0.13ms        ? ?/sec    1.02     42.0±0.10ms        ? ?/sec
last_value merge_bench list<struct(i64,utf8)>[4] nulls=90%                    1.00     41.6±0.14ms        ? ?/sec    1.03     42.7±0.19ms        ? ?/sec
last_value merge_bench map<utf8,i64> nulls=0%                                 1.00     40.5±0.12ms        ? ?/sec    1.03     41.6±0.16ms        ? ?/sec
last_value merge_bench map<utf8,i64> nulls=90%                                1.00     41.9±0.15ms        ? ?/sec    1.03     43.0±0.13ms        ? ?/sec
last_value merge_bench nulls=0%, filter=false                                 1.00     35.9±0.09ms        ? ?/sec    1.01     36.5±0.17ms        ? ?/sec
last_value merge_bench nulls=0%, filter=true                                  1.00     36.0±0.10ms        ? ?/sec    1.01     36.1±0.11ms        ? ?/sec
last_value merge_bench nulls=90%, filter=false                                1.00     46.3±0.08ms        ? ?/sec    1.01     46.7±0.12ms        ? ?/sec
last_value merge_bench nulls=90%, filter=true                                 1.00     46.7±0.09ms        ? ?/sec    1.01     47.0±0.16ms        ? ?/sec
last_value merge_bench struct(i64,utf8,f64) nulls=0%                          1.00     40.4±0.19ms        ? ?/sec    1.03     41.6±0.13ms        ? ?/sec
last_value merge_bench struct(i64,utf8,f64) nulls=90%                         1.00     42.6±0.11ms        ? ?/sec    1.02     43.5±0.19ms        ? ?/sec
last_value trivial_update_bench nulls=0%, ignore_nulls=false                  1.00      2.6±0.05µs        ? ?/sec    1.00      2.6±0.05µs        ? ?/sec
last_value trivial_update_bench nulls=0%, ignore_nulls=true                   1.00      2.7±0.05µs        ? ?/sec    1.01      2.7±0.05µs        ? ?/sec
last_value trivial_update_bench nulls=90%, ignore_nulls=false                 1.01      3.0±0.03µs        ? ?/sec    1.00      2.9±0.03µs        ? ?/sec
last_value trivial_update_bench nulls=90%, ignore_nulls=true                  1.00      6.7±0.04µs        ? ?/sec    1.09      7.3±0.06µs        ? ?/sec
last_value update_bench list<i64>[4] nulls=0%                                 1.00     33.3±0.13ms        ? ?/sec    1.00     33.2±0.10ms        ? ?/sec
last_value update_bench list<i64>[4] nulls=90%                                1.00     33.6±0.10ms        ? ?/sec    1.01     34.0±0.15ms        ? ?/sec
last_value update_bench list<struct(i64,utf8)>[4] nulls=0%                    1.01     36.5±0.13ms        ? ?/sec    1.00     36.3±0.10ms        ? ?/sec
last_value update_bench list<struct(i64,utf8)>[4] nulls=90%                   1.00     37.2±0.12ms        ? ?/sec    1.00     37.3±0.15ms        ? ?/sec
last_value update_bench map<utf8,i64> nulls=0%                                1.00     36.0±0.11ms        ? ?/sec    1.00     36.1±0.13ms        ? ?/sec
last_value update_bench map<utf8,i64> nulls=90%                               1.00     37.5±0.10ms        ? ?/sec    1.00     37.6±0.20ms        ? ?/sec
last_value update_bench nulls=0%, filter=false                                1.01     30.7±0.08ms        ? ?/sec    1.00     30.5±0.12ms        ? ?/sec
last_value update_bench nulls=0%, filter=true                                 1.05     47.1±0.08ms        ? ?/sec    1.00     44.8±0.09ms        ? ?/sec
last_value update_bench nulls=90%, filter=false                               1.01     42.3±0.07ms        ? ?/sec    1.00     42.0±0.12ms        ? ?/sec
last_value update_bench nulls=90%, filter=true                                1.02     53.6±0.15ms        ? ?/sec    1.00     52.8±0.14ms        ? ?/sec
last_value update_bench struct(i64,utf8,f64) nulls=0%                         1.00     35.8±0.15ms        ? ?/sec    1.00     36.0±0.10ms        ? ?/sec
last_value update_bench struct(i64,utf8,f64) nulls=90%                        1.00     38.2±0.11ms        ? ?/sec    1.00     38.1±0.11ms        ? ?/sec

Resource Usage

first_last — base (merge-base)

Metric	Value
Wall time	1390.3s
Peak memory	2.8 GiB
Avg memory	112.9 MiB
CPU user	1234.8s
CPU sys	9.7s
Peak spill	0 B

first_last — branch

Metric	Value
Wall time	1365.3s
Peak memory	2.8 GiB
Avg memory	115.2 MiB
CPU user	1232.5s
CPU sys	9.6s
Peak spill	0 B

---

File an issue against this benchmark runner

[tanmay442]

@tanmay442 can you please not paste the whole llm response its hard to figure out what's important from the wall of text. See https://datafusion.apache.org/contributor-guide/ai-policy.html

@kumarUjjawal
Hey, I am really sorry for the discomfort caused. It was really my neglect, and I shouldn't have done that. I see that these new changes also have no performance gains and only regressions. I agree it is my fault; I just read the code initially and thought of a fix, but I definitely severely underestimated the complexity of the project. And since after the first bench run (which was a degradation) I felt like I had to fix it, which I did not know how to do, so I threw Opus at it, which I shouldn't have done. I take the blame and understand this is a bad practice, truly apologizing for the waste of your time and mental resources. I would like to call it here and close this PR sincerely, and the other one as well, since they serve no purpose to anyone and are honestly slop. As someone who has recently learned the language to a competent enough extent, I thought exploring large codebases should be the next step, but was I wrong . Once again, I apologize and will make sure that I never spam any OSS project ever again. Trully appreciate your patience , thanks