chore(deps): drop redundant transitive pins from requirements#6110
Conversation
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6110 +/- ##
============================================
- Coverage 70.44% 70.42% -0.03%
Complexity 3404 3404
============================================
Files 1142 1142
Lines 44876 44876
Branches 4955 4955
============================================
- Hits 31613 31602 -11
- Misses 11619 11632 +13
+ Partials 1644 1642 -2
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 378 | 0.231 | 25,425/36,747/36,747 us | 🔴 +13.5% / 🔴 +155.9% |
| 🔴 | bs=100 sw=10 sl=64 | 946 | 0.577 | 102,629/126,950/126,950 us | 🔴 +8.8% / 🔴 +28.1% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,109 | 0.677 | 892,772/965,837/965,837 us | ⚪ within ±5% / ⚪ within ±5% |
Baseline details
Latest main f722c26 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 378 tuples/sec | 424 tuples/sec | 831.47 tuples/sec | -10.8% | -54.5% |
| bs=10 sw=10 sl=64 | MB/s | 0.231 MB/s | 0.259 MB/s | 0.507 MB/s | -10.8% | -54.5% |
| bs=10 sw=10 sl=64 | p50 | 25,425 us | 22,395 us | 11,844 us | +13.5% | +114.7% |
| bs=10 sw=10 sl=64 | p95 | 36,747 us | 36,536 us | 14,359 us | +0.6% | +155.9% |
| bs=10 sw=10 sl=64 | p99 | 36,747 us | 36,536 us | 18,888 us | +0.6% | +94.6% |
| bs=100 sw=10 sl=64 | throughput | 946 tuples/sec | 998 tuples/sec | 1,089 tuples/sec | -5.2% | -13.1% |
| bs=100 sw=10 sl=64 | MB/s | 0.577 MB/s | 0.609 MB/s | 0.665 MB/s | -5.3% | -13.2% |
| bs=100 sw=10 sl=64 | p50 | 102,629 us | 99,079 us | 92,503 us | +3.6% | +10.9% |
| bs=100 sw=10 sl=64 | p95 | 126,950 us | 116,732 us | 99,121 us | +8.8% | +28.1% |
| bs=100 sw=10 sl=64 | p99 | 126,950 us | 116,732 us | 112,422 us | +8.8% | +12.9% |
| bs=1000 sw=10 sl=64 | throughput | 1,109 tuples/sec | 1,108 tuples/sec | 1,117 tuples/sec | +0.1% | -0.7% |
| bs=1000 sw=10 sl=64 | MB/s | 0.677 MB/s | 0.676 MB/s | 0.682 MB/s | +0.1% | -0.7% |
| bs=1000 sw=10 sl=64 | p50 | 892,772 us | 899,666 us | 910,046 us | -0.8% | -1.9% |
| bs=1000 sw=10 sl=64 | p95 | 965,837 us | 954,393 us | 950,716 us | +1.2% | +1.6% |
| bs=1000 sw=10 sl=64 | p99 | 965,837 us | 954,393 us | 985,832 us | +1.2% | -2.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,528.85,200,128000,378,0.231,25424.75,36747.48,36747.48
1,100,10,64,20,2114.23,2000,1280000,946,0.577,102628.90,126950.49,126950.49
2,1000,10,64,20,18027.74,20000,12800000,1109,0.677,892772.30,965837.40,965837.40|
cc @SarahAsad23 for review |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
f3f7acf to
89eb663
Compare
|
@aicam @mengw15 and @SarahAsad23 please help on review this. thanks |
Signed-off-by: Yicong Huang <17627829+Yicong-Huang@users.noreply.github.com>
Signed-off-by: Yicong Huang <17627829+Yicong-Huang@users.noreply.github.com>
|
LGTM! |
mengw15
left a comment
There was a problem hiding this comment.
Left one comment on the failing license gate
Signed-off-by: Yicong Huang <17627829+Yicong-Huang@users.noreply.github.com>
… resolved version python-dateutil is still installed transitively (pandas, botocore), so the license gate requires it to stay claimed; update the pin-less resolved version to 2.9.0.post0 instead of removing the line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
What changes were proposed in this PR?
Audits the explicit transitive pins in
amber/requirements.txtand removes the two that serve no purpose. Verified with the pip dependency graph (Required-by) and a fullpip install --dry-run --reportresolution:python-dateutil==2.8.2pip freezeleftover from the original requirements.txt (#1259); every dependent declares>=2.8, so the pin only held us on a 2021 release. Resolves to 2.9.0.post0aiobotocore==3.7.0botocore==1.42.90pin (aiobotocore's per-release botocore window is very narrow). Removing it eliminates a manual pairing hazard when bumping the boto3/botocore pins. Still resolves to 3.7.0s3fs,SQLAlchemy,pg8000Required-by— pyiceberg only pulls them via extras, so dropping them would actually uninstall them (Iceberg S3 FileIO / SqlCatalog would break at runtime). pg8000 deliberately replaces LGPL psycopg2 (#3299)setuptools==80.10.2fs'spkg_resourcesimport against newer setuptools removing it (#4199);fsdeclares setuptools with no version boundamber/LICENSE-binary-pythonupdatespython-dateutilto the resolved 2.9.0.post0 (it stays installed transitively, so thecheck_binary_deps.pygate still requires the claim); aiobotocore already claims 3.7.0, which is unchanged by the resolution.Any related issues, documentation, discussions?
Closes #6108. Follow-up to #6101 / #6103.
How was this PR tested?
pip install --dry-run --ignore-installed --reportwithout the pins: resolution unchanged except python-dateutil → 2.9.0.post0; aiobotocore/botocore/boto3 stay at their current versions.pytest -m "not integration") — passes.check_binary_deps.pyverifies the LICENSE-binary-python claims.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-fable-5)
🤖 Generated with Claude Code