[Superseded] perf: vectorize small u8 table take

[joseph-isaacs]

Rationale

Low-cardinality dictionaries with u8 codes and one-byte values can decode through an in-register table instead of performing arbitrary scalar lookups.

Stacked on #9566, which adds the benchmark-only baseline. Keeping the implementation in this upper PR lets CodSpeed report the wall-time delta directly.

Changes

• Use NEON TBL for AArch64.
• Use AVX2 VPSHUFB for x86/x86-64.
• Apply the specialized path to u8 codes, at most 16 one-byte values, and at least 64 rows.
• Retain bounds checks and the existing scalar/general fallbacks.
• Add correctness and out-of-bounds tests.

CodSpeed wall-time results

All figures are medians over 1,000 samples on the CPU-feature metal legs. Baseline: #9566.

Build	Rows	Baseline	This PR	Speedup	Time reduction
AVX2	1M	428.1 µs	61.12 µs	7.00×	85.7%
AVX2	16M	6.809 ms	1.361 ms	5.00×	80.0%
AVX-512	1M	426.5 µs	59.28 µs	7.19×	86.1%
AVX-512	16M	6.778 ms	1.358 ms	4.99×	80.0%
NEON	1M	552.4 µs	51.06 µs	10.82×	90.8%
NEON	16M	8.786 ms	799.7 µs	10.99×	90.9%

Runs: baseline, optimized.

Validation

• 3,352 vortex-array tests passed; 1 skipped
• targeted Clippy passed
• x86-64 cross-check passed
• formatting and diff checks passed

[codspeed-hq]

Merging this PR will regress 3 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 9 improved benchmarks
❌ 3 regressed benchmarks
✅ 1969 untouched benchmarks
🆕 6 new benchmarks
⏩ 54 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	words_gather_scalar_avx2[65536]	8.3 µs	9.4 µs	-12.35%
❌	Simulation	compact[(1024, 90)]	1.6 µs	1.8 µs	-10.8%
❌	Simulation	compact[(2048, 90)]	1.7 µs	1.9 µs	-10.4%
⚡	Simulation	decode_primitives[u8, (2000, 4)]	51.4 µs	41.1 µs	+25.06%
⚡	Simulation	decode_primitives[u8, (2000, 2)]	51.5 µs	41.3 µs	+24.82%
⚡	Simulation	decode_primitives[u8, (2000, 8)]	51.9 µs	41.7 µs	+24.39%
⚡	Simulation	take[small_m/shuffled/primitive/nonnull/chunks=2048/indices=64]	801.9 µs	711.9 µs	+12.64%
⚡	Simulation	decode_primitives[u8, (1000, 8)]	43.5 µs	38.8 µs	+12.16%
⚡	Simulation	decode_primitives[u8, (1000, 4)]	43.7 µs	39 µs	+12.12%
⚡	Simulation	take[small_m/shuffled/primitive/nonnull/chunks=256/indices=64]	611.3 µs	545.4 µs	+12.07%
⚡	Simulation	decode_primitives[u8, (1000, 2)]	44.6 µs	39.8 µs	+12.04%
⚡	Simulation	take[small_m/shuffled/primitive/nonnull/chunks=2048/indices=16]	376.2 µs	338.5 µs	+11.16%
🆕	WallTime	dict_canonicalize_gt_u8_neon[1000000]	N/A	50.7 µs	N/A
🆕	WallTime	dict_canonicalize_gt_u8_neon[16000000]	N/A	779.5 µs	N/A
🆕	WallTime	dict_canonicalize_gt_u8_avx2[1000000]	N/A	60 µs	N/A
🆕	WallTime	dict_canonicalize_gt_u8_avx2[16000000]	N/A	1.4 ms	N/A
🆕	WallTime	dict_canonicalize_gt_u8_avx512[1000000]	N/A	57.2 µs	N/A
🆕	WallTime	dict_canonicalize_gt_u8_avx512[16000000]	N/A	1.3 ms	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ji/small-u8-table-take (593e136) with develop (ef43456)²}

Footnotes

1. 54 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on ji/small-u8-table-take-benchmark (d49393c) during the generation of this report, so develop (ef43456) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩