Repository navigation
Conversation
Contributor
Author
|
Hey @alexey-milovidov can you please take a look? |
alexey-milovidov
added
the
reproduction-pending
Contributor
Author
|
I saw this is marked as reproduction-pending, how can we help with that? |
Fresh trial account, unmodified upstream harness, client in AWS us-east-1. Each size was switched via the REST API and given a 5-minute settle plus 3 consecutive md_live_duckling_size() confirmations before running, which fixes the slow standard/jumbo load times in the earlier runs. Adds pulse, and tags all results as "DuckDB derivative" like the existing entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
jtigani
changed the title
Contributor
Author
|
I used claude to run this against all motherduck instance sizes in a clean trial account (the only thing I had do do on the backend was enable the largest size, which is not typically available for trial customers until they convert to paid). |
jtigani
added a commit
to jtigani/ClickBench
that referenced
this pull request
Oct 20, 2026
Each query try opens a fresh md: connection. Connect with
config={"motherduck_attach_mode": "single"} so it attaches only the database the
benchmark uses instead of enumerating the whole workspace catalog on every
connect. Verified via the config dict (the connection-string form is silently
ignored); queries still return correct results.
Context: the geomean comparison vs PR ClickHouse#2319 showed the migrated entry carries a
small, systematic per-query overhead from the one-connection-per-try model
(worst on fast queries / cold runs). Single-attach trims the catalog work of
that per-try connect. A local A/B was inconclusive (in-process connection
pooling + cross-country latency mask the fresh-process cost); the real effect
should be measured on the co-located EC2 client.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the 20261016 standard/jumbo/mega/giga results with re-runs from 2026-10-26/27, using the shared-runner MotherDuck entry from ClickHouse#2702 (DuckDB 1.5.6 client) from EC2 in us-east-1. Pulse keeps its 20261016 result. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
jtigani
changed the title
Contributor
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding recent MotherDuck runs, realized our data was almost a year old. Should be a good reset before duckdb 2.0 comes out in a few weeks.
Updated 2026-10-27: standard, jumbo, mega and giga were re-run with the shared-runner MotherDuck entry from #2702, replacing their 2026-10-16 results. Pulse keeps its 2026-10-16 result.
20261016/pulse.json20261027/standard.json20261026/jumbo.json20261027/mega.json20261026/giga.jsonHow these were run:
aws-us-east-1, client on EC2 (c6a.4xlarge) in us-east-1.motherduck/run-all-sizes.shfrom MotherDuck: migrate to the shared benchmark runner #2702, with the DuckDB client pinned to 1.5.6 (the latest version MotherDuck supports; the unpinned install now pulls 2.0.0). Pulse: unmodifiedmotherduck/benchmark.shfrommain.PUT /v1/users/{user}/instances), and the run only started oncemd_live_duckling_size()reported the new size.DuckDB derivative, like the existing MotherDuck entries.🤖 Generated with Claude Code