From 6a8b4a4eee74eede7c6c199d3db4490fdae88c8d Mon Sep 17 00:00:00 2001 From: Panagiotis Moustafellos Date: Mon, 26 Oct 2026 04:09:59 +0200 Subject: [PATCH 1/5] ravel: run the entry's object store on s3s-fs instead of RustFS Ravel reads cold statements as ranged GETs. On a single drive RustFS serves each ranged GET by reading the whole 1 MiB erasure block around it, about 1.5 MB of disk per 10 KB range measured on c6a.4xlarge, so the cold figures measured the store's read amplification more than the database. s3s-fs serves a range with one file read. Both are single-node S3 servers over the machine's disk; the entry stays a local-disk entry with nothing required from the operator. ./install downloads the s3s-fs binary for the host architecture from a release on the NOFireAI/s3s fork (s3s-fs-0.17.0-ravel.1: the v0.17.0 release plus listing ETags, s3s-project/s3s#1013, which Ravel's store qualification requires, and keys read from the environment, s3s-project/s3s#1014, so no key sits in a process argument list) and verifies it against that release's SHA256SUMS before moving it into place, the way Ravel's own binaries are installed. When the release is unreachable it builds the same pinned commit with cargo install --locked, so what runs is fixed by content either way; the commit installed is recorded beside the binary. RustFS is gone from the entry: nothing downloads it. rustfs-start becomes store-start, the store directory is store/ and the generated keys are STORE_ACCESS_KEY and STORE_SECRET_KEY. The README gains a section on the store that states the swap and the measured reason for it. --- ravel/.gitignore | 4 +- ravel/README.md | 91 ++++++++++++++++++++++------------ ravel/benchmark.sh | 4 +- ravel/install | 119 +++++++++++++++++++++++++++++---------------- ravel/ravel-env.sh | 27 +++++----- ravel/rustfs-start | 45 ----------------- ravel/start | 4 +- ravel/store-start | 47 ++++++++++++++++++ 8 files changed, 202 insertions(+), 139 deletions(-) delete mode 100755 ravel/rustfs-start create mode 100755 ravel/store-start diff --git a/ravel/.gitignore b/ravel/.gitignore index 3ac91f9c5d..b87a9478dc 100644 --- a/ravel/.gitignore +++ b/ravel/.gitignore @@ -8,6 +8,6 @@ server.pid *.log hits.parquet cache/ -# The local RustFS ./install sets up: binary, data directory, pid, log and the +# The local s3s-fs ./install sets up: binary, data directory, pid, log and the # generated credentials. -rustfs/ +store/ diff --git a/ravel/README.md b/ravel/README.md index 3464397fe8..41eedc3564 100644 --- a/ravel/README.md +++ b/ravel/README.md @@ -8,7 +8,7 @@ is loaded as the logs signal, with each column declared as a typed attribute column, and queried over SQL. For this benchmark the object store is a single-node -[RustFS](https://github.com/rustfs/rustfs) that `./install` downloads, starts +[s3s-fs](https://github.com/s3s-project/s3s) that `./install` sets up, starts and provisions on the machine's own disk, so the entry runs like the other local-disk systems in this repository and needs no cloud account. Run it the usual way: @@ -19,38 +19,63 @@ usual way: ## What this entry needs -Nothing from the operator. `./install` fetches a pinned RustFS release from -GitHub, verifies it against the release's `SHA256SUMS`, generates a credential -pair into a file readable by the current user only (`rustfs/credentials.env`), -starts RustFS on loopback with its data directory under `rustfs/data`, creates -the bucket, and runs `ravel-cli store qualify` once, the bucket's one-time -conformance check (conditional writes, read-after-write and list-after-write -consistency, listing order). The first `./start` bootstraps the bucket's -`sys/tenancy` marker to the unkeyed tenant-hash derivation: the server's own -default is the keyed derivation, which refuses to start without a deployment -key file, and this benchmark holds no secret and needs a derivable prefix for -`./data-size`. - -Credentials reach RustFS and Ravel through the environment, never a command +Nothing from the operator. `./install` downloads a prebuilt s3s-fs for the +host architecture from a release on our fork and verifies it against that +release's `SHA256SUMS` (if the release is unreachable it builds the same +pinned commit with `cargo install --locked`, installing a minimal Rust +toolchain if the machine has none), generates a credential pair into a file +readable by the current user only (`store/credentials.env`), starts s3s-fs on +loopback with its data +directory under `store/data`, creates the bucket, and runs `ravel-cli store +qualify` once, the bucket's one-time conformance check (conditional writes, +read-after-write and list-after-write consistency, listing order). The first +`./start` bootstraps the bucket's `sys/tenancy` marker to the unkeyed +tenant-hash derivation: the server's own default is the keyed derivation, +which refuses to start without a deployment key file, and this benchmark holds +no secret and needs a derivable prefix for `./data-size`. + +Credentials reach s3s-fs and Ravel through the environment, never a command line; `ravel-env.sh` reads them from the generated file. The same file holds the key the server tokenizes query text with on its audit trail (`RAVEL_AUDIT_TOKEN_KEY`, generated by `./install` too): a keyed deployment derives that key, and this entry runs unkeyed, so it supplies one of its own. -RustFS's web console is disabled, so the entry opens two loopback ports, the -store's and the database's. `./stop` stops the database only. RustFS stays up -across the driver's restart cycle and `./rustfs-start` restarts it if it is -ever found down, so the drop of the page cache before each cold run reaches -RustFS's files the same way it reaches any local-disk engine's. +The entry opens two loopback ports, the store's and the database's. `./stop` +stops the database only. s3s-fs stays up across the driver's restart cycle and +`./store-start` restarts it if it is ever found down, so the drop of the page +cache before each cold run reaches the store's files the same way it reaches +any local-disk engine's. + +### The object store + +The store is s3s-fs, the file-system backend of the s3s project, instead of +RustFS, which earlier versions of this entry used. The binary comes from a +release on [our fork](https://github.com/NOFireAI/s3s), built from a pinned +commit and verified against the release's `SHA256SUMS`: the s3s-fs +v0.17.0 release plus two changes offered upstream, one that returns each +object's stored ETag in listings +([s3s-project/s3s#1013](https://github.com/s3s-project/s3s/pull/1013)), which +Ravel's store qualification requires, and one that reads the access and secret +key from the environment instead of the argument list +([s3s-project/s3s#1014](https://github.com/s3s-project/s3s/pull/1014)). Both +are single-node S3 servers over this machine's disk; the difference is what a +ranged read costs. On a single drive RustFS reads a whole 1 MiB erasure block +around every ranged GET (about 1.5 MB of disk per 10 KB range, measured on +c6a.4xlarge), while s3s-fs serves a range with one file read. Ravel's cold +reads are ranged, so on RustFS the cold figures measured the store's read +amplification more than the database; see "What the cold figures measure +here" below. To run the same scripts against another S3-compatible store, set `RAVEL_S3_ENDPOINT`, `RAVEL_S3_BUCKET`, `RAVEL_S3_ACCESS_KEY` and `RAVEL_S3_SECRET_KEY` before `./install`; a non-loopback endpoint skips the -RustFS setup. `RAVEL_S3_REGION` defaults to `us-east-1`. +s3s-fs setup. `RAVEL_S3_REGION` defaults to `us-east-1`. Optional: `RAVEL_TENANT` (default `clickbench`), `RAVEL_SHARDS` (default 4, must match between `./load` and `./start`), `RAVEL_VERSION` (default the released version `./install` downloads), `RAVEL_REF` (build that ref from -source instead), `RUSTFS_VERSION` (the pinned RustFS release). +source instead), `S3S_FS_GIT`, `S3S_FS_RELEASE` and `S3S_FS_REV` (the +repository, the release the s3s-fs binary is downloaded from, and the commit +it is built from when that release is unreachable). ## How the run is configured @@ -108,17 +133,19 @@ Ravel's state lives. The reference machine's disk is a 500 GB gp2 volume. Ravel holds no data on local disk, so a cold run (server restarted, page cache dropped) fetches what it -needs from RustFS, and RustFS reads it from that volume. Since 0.24.0, a server -whose store is on loopback defaults its fetch policy to `latency-first`, which -reads only the byte ranges a statement needs from each object instead of whole -objects; `server.log` names it with `policy_source="default-loopback"`. On -c6a.4xlarge, in this repository's fresh-VM setup, the cold sum over the 43 -statements was 365.7 s on v0.26.0, against 486.6 s on v0.23.0 and 1,291.0 s on -v0.21.0. The dataset is 10.2 GB. - -A warm run is served from the read cache or from RustFS. Since 0.24.0, a server -whose store is on loopback gives its fetch cache 24% of its memory budget: 7.0 -GB on this machine. +needs from the store, and the store reads it from that volume. Since 0.24.0, a +server whose store is on loopback defaults its fetch policy to `latency-first`, +which reads only the byte ranges a statement needs from each object instead of +whole objects; `server.log` names it with `policy_source="default-loopback"`. +On c6a.4xlarge, in this repository's fresh-VM setup, with RustFS as the store, +the cold sum over the 43 statements was 365.7 s on v0.26.0, against 486.6 s on +v0.23.0 and 1,291.0 s on v0.21.0. The dataset is 10.2 GB. The published +result in `results/` was measured with RustFS; a run of this entry as it +stands, with s3s-fs, replaces it. + +A warm run is served from the read cache or from the store. Since 0.24.0, a +server whose store is on loopback gives its fetch cache 24% of its memory +budget: 7.0 GB on this machine. ### The tuned configuration diff --git a/ravel/benchmark.sh b/ravel/benchmark.sh index ebd6651935..7741f88c4f 100755 --- a/ravel/benchmark.sh +++ b/ravel/benchmark.sh @@ -2,8 +2,8 @@ # Ravel: object-storage-native telemetry database, queried over SQL. # # Ravel keeps every durable byte in S3-compatible object storage; there is no -# local-disk storage mode. For this benchmark the store is a single-node RustFS -# that ./install downloads, starts and provisions on this machine's disk, with +# local-disk storage mode. For this benchmark the store is a single-node s3s-fs +# that ./install sets up, starts and provisions on this machine's disk, with # credentials it generates into a file readable by the current user only, so # nothing is required from the operator and no key is stored in this # repository. See README.md. diff --git a/ravel/install b/ravel/install index a0c4350a7b..aecba56662 100755 --- a/ravel/install +++ b/ravel/install @@ -53,6 +53,23 @@ esac mkdir -p "$RAVEL_BIN_DIR" +# A Rust toolchain and a C linker: always for the store below, and for the +# source build of Ravel when its release is unreachable. +ensure_cargo() { + if command -v apt-get >/dev/null 2>&1; then + sudo apt-get install -y --no-install-recommends git build-essential pkg-config libssl-dev protobuf-compiler + else + sudo dnf install -y -q git gcc gcc-c++ make pkgconfig openssl-devel protobuf-compiler + fi + if ! cargo --version >/dev/null 2>&1; then + if command -v rustup >/dev/null 2>&1; then rustup default stable + else curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal; fi + fi + # shellcheck disable=SC1091 + [ -f "$HOME/.cargo/env" ] && . "$HOME/.cargo/env" + cargo --version +} + # Preferred path: the released binaries, which are the ones extracted from the # signed container images, so a downloaded binary is byte-identical to what the # published image runs (ADR-0086). Checksums are verified against the release's @@ -78,17 +95,7 @@ else # Fallback: build at a ref. Used when the release is unreachable, or to # benchmark an unreleased commit via RAVEL_REF. echo "install: release ${RAVEL_VERSION} not reachable, building from source" >&2 - if command -v apt-get >/dev/null 2>&1; then - sudo apt-get install -y --no-install-recommends git build-essential pkg-config libssl-dev protobuf-compiler - else - sudo dnf install -y -q git gcc gcc-c++ make pkgconfig openssl-devel protobuf-compiler - fi - if ! cargo --version >/dev/null 2>&1; then - if command -v rustup >/dev/null 2>&1; then rustup default stable - else curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal; fi - fi - # shellcheck disable=SC1091 - [ -f "$HOME/.cargo/env" ] && . "$HOME/.cargo/env" + ensure_cargo [ -d src ] || git clone --filter=blob:none "${RAVEL_REPO}.git" src git -C src fetch origin --quiet git -C src checkout -q --detach "${RAVEL_REF:-$RAVEL_VERSION}" @@ -103,62 +110,88 @@ fi # What will run, by content rather than by a version string. sha256sum "$RAVEL_SERVER" "$RAVEL_CLI" -# The object store: a single-node RustFS on this machine's disk, so the entry +# The object store: a single-node s3s-fs on this machine's disk, so the entry # is comparable with the local-disk systems in this repository and needs no -# cloud account. A pinned release from GitHub, verified against the release's -# own SHA256SUMS. A loopback RAVEL_S3_ENDPOINT means the RustFS this entry -# manages; anything else is an external S3-compatible store the operator -# provisioned, and this section is skipped. -RUSTFS_VERSION="${RUSTFS_VERSION:-1.0.0}" +# cloud account. s3s-fs is the file-system backend of the s3s project +# (https://github.com/s3s-project/s3s). The binary comes from a release on our +# fork, verified against that release's own SHA256SUMS the way Ravel's binaries +# are above: the v0.17.0 release plus two changes offered upstream, one that +# returns each object's stored ETag in listings (s3s-project/s3s#1013), which +# `store qualify` requires, and one that reads the access and secret key from +# the environment (s3s-project/s3s#1014). The release is built from the commit +# S3S_FS_REV names; when the release is unreachable the same commit is built +# here with `cargo install --locked`, so what runs is fixed by content either +# way. A loopback RAVEL_S3_ENDPOINT means the s3s-fs this entry manages; +# anything else is an external S3-compatible store the operator provisioned, +# and this section is skipped. +S3S_FS_GIT="${S3S_FS_GIT:-https://github.com/NOFireAI/s3s}" +S3S_FS_RELEASE="${S3S_FS_RELEASE:-s3s-fs-0.17.0-ravel.1}" +S3S_FS_REV="${S3S_FS_REV:-9353c1f1b3d4247d1d48858e34abcb21e2003ac5}" case "$RAVEL_S3_ENDPOINT" in http://127.0.0.1:*|http://localhost:*) local_store=yes ;; *) local_store=no ;; esac if [ "$local_store" = yes ]; then - mkdir -p "$RUSTFS_DIR" - if ! [ -x "$RUSTFS_DIR/rustfs" ] || ! "$RUSTFS_DIR/rustfs" --version 2>/dev/null | grep -q "^rustfs ${RUSTFS_VERSION}\$"; then + mkdir -p "$STORE_DIR/bin" + # Fetched or built only when the pin moves: the version string does not + # carry the commit, so the commit installed is recorded beside the binary. + if ! [ -x "$STORE_DIR/bin/s3s-fs" ] || [ "$(cat "$STORE_DIR/s3s-fs.rev" 2>/dev/null)" != "$S3S_FS_REV" ]; then case "$arch" in - amd64) rarch=x86_64 ;; - arm64) rarch=aarch64 ;; + amd64) sarch=x86_64 ;; + arm64) sarch=aarch64 ;; esac - zip="rustfs-linux-${rarch}-gnu-v${RUSTFS_VERSION}.zip" - rbase="https://github.com/rustfs/rustfs/releases/download/${RUSTFS_VERSION}" - curl -fsSL --retry 3 -o "$RUSTFS_DIR/$zip" "${rbase}/${zip}" - want=$(curl -fsSL --retry 3 "${rbase}/SHA256SUMS" | awk -v n="$zip" '$2 == n {print $1}') - have=$(sha256sum "$RUSTFS_DIR/$zip" | awk '{print $1}') - [ -n "$want" ] && [ "$want" = "$have" ] || { echo "install: checksum mismatch for $zip" >&2; exit 1; } - rm -rf "$RUSTFS_DIR/unpack" - unzip -q -o "$RUSTFS_DIR/$zip" -d "$RUSTFS_DIR/unpack" - chmod +x "$RUSTFS_DIR/unpack/rustfs" - # A running RustFS keeps serving from the replaced inode; ./rustfs-start - # only launches a new one when the health check fails. - mv -f "$RUSTFS_DIR/unpack/rustfs" "$RUSTFS_DIR/rustfs" - rm -rf "$RUSTFS_DIR/unpack" "$RUSTFS_DIR/$zip" + sbase="${S3S_FS_GIT}/releases/download/${S3S_FS_RELEASE}" + if curl -fsSL --retry 3 -o "$STORE_DIR/SHA256SUMS" "${sbase}/SHA256SUMS"; then + # Downloaded under a temporary name and moved into place once the + # checksum passes, so an unverified binary never sits at the path + # ./store-start runs. A running s3s-fs keeps serving from the + # replaced inode; ./store-start only launches a new one when the + # endpoint stops answering. + curl -fsSL --retry 3 -o "$STORE_DIR/bin/s3s-fs.new" "${sbase}/s3s-fs-linux-${sarch}" + want=$(awk -v n="s3s-fs-linux-${sarch}" '$2 == n || $2 == "*"n {print $1}' "$STORE_DIR/SHA256SUMS" | head -1) + [ -n "$want" ] || { echo "install: s3s-fs-linux-${sarch} absent from the release's SHA256SUMS" >&2; exit 1; } + have=$(sha256sum "$STORE_DIR/bin/s3s-fs.new" | awk '{print $1}') + [ "$want" = "$have" ] || { echo "install: checksum mismatch for s3s-fs-linux-${sarch}" >&2; exit 1; } + chmod +x "$STORE_DIR/bin/s3s-fs.new" + mv -f "$STORE_DIR/bin/s3s-fs.new" "$STORE_DIR/bin/s3s-fs" + echo "s3s-fs release ${S3S_FS_RELEASE}" + else + # Fallback: build the same commit. Measured on this repository's + # image: about 3 minutes and 1.1 GB of RSS on c6a.large, 30 s on + # c8g.4xlarge. + echo "install: release ${S3S_FS_RELEASE} not reachable, building s3s-fs from source" >&2 + ensure_cargo + cargo install --locked --git "$S3S_FS_GIT" --rev "$S3S_FS_REV" s3s-fs --features binary --root "$STORE_DIR" + echo "s3s-fs built from source" + fi + echo "$S3S_FS_REV" > "$STORE_DIR/s3s-fs.rev" fi - "$RUSTFS_DIR/rustfs" --version | head -n 1 + "$STORE_DIR/bin/s3s-fs" --version | head -n 1 + echo "s3s-fs commit $(cat "$STORE_DIR/s3s-fs.rev")" + sha256sum "$STORE_DIR/bin/s3s-fs" # Credentials: generated once, readable by this user only, handed to - # RustFS and to Ravel through the environment. Nothing in this repository + # s3s-fs and to Ravel through the environment. Nothing in this repository # or in a process argument list carries a key. - if [ ! -f "$RUSTFS_DIR/credentials.env" ]; then + if [ ! -f "$STORE_DIR/credentials.env" ]; then ( umask 077 - printf 'RUSTFS_ACCESS_KEY=%s\nRUSTFS_SECRET_KEY=%s\n' \ + printf 'STORE_ACCESS_KEY=%s\nSTORE_SECRET_KEY=%s\n' \ "ravel-$(head -c 6 /dev/urandom | od -An -tx1 | tr -d ' \n')" \ "$(head -c 30 /dev/urandom | base64 | tr -d '/+=\n')" \ - > "$RUSTFS_DIR/credentials.env" ) + > "$STORE_DIR/credentials.env" ) fi # The server records query text on its audit trail tokenized under a # key. A keyed deployment derives it; this entry runs unkeyed (see # ./start), so it needs one of its own. Generated once, kept beside the - # RustFS credentials, handed to the server through the environment. - if ! grep -q '^RAVEL_AUDIT_TOKEN_KEY=' "$RUSTFS_DIR/credentials.env"; then + # store credentials, handed to the server through the environment. + if ! grep -q '^RAVEL_AUDIT_TOKEN_KEY=' "$STORE_DIR/credentials.env"; then ( umask 077 printf 'RAVEL_AUDIT_TOKEN_KEY=%s\n' \ "$(head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n')" \ - >> "$RUSTFS_DIR/credentials.env" ) + >> "$STORE_DIR/credentials.env" ) fi # Re-source so the keys just written are exported for the steps below. . ./ravel-env.sh - ./rustfs-start + ./store-start fi # The bucket. Everything Ravel writes lands under a t// prefix in diff --git a/ravel/ravel-env.sh b/ravel/ravel-env.sh index 8087716edf..8897b186f9 100644 --- a/ravel/ravel-env.sh +++ b/ravel/ravel-env.sh @@ -2,7 +2,7 @@ # # Ravel keeps every durable byte in S3-compatible object storage; there is no # local-disk storage mode (see README.md). For this benchmark the store is a -# single-node RustFS that ./install downloads, starts and provisions on this +# single-node s3s-fs that ./install sets up, starts and provisions on this # machine's own disk, so the entry needs no cloud bucket and no credentials # from the operator. To run against another S3-compatible store instead, set # RAVEL_S3_ENDPOINT, RAVEL_S3_BUCKET, RAVEL_S3_ACCESS_KEY and @@ -13,27 +13,28 @@ export RAVEL_S3_ENDPOINT="${RAVEL_S3_ENDPOINT:-http://127.0.0.1:9000}" export RAVEL_S3_BUCKET="${RAVEL_S3_BUCKET:-clickbench}" export RAVEL_S3_REGION="${RAVEL_S3_REGION:-us-east-1}" -# Where ./install keeps RustFS: its binary, its data directory (this entry's -# "data on local disk"), its pid file, its log and the credentials it -# generated. ./rustfs-start brings it up from here. -export RUSTFS_DIR="${RUSTFS_DIR:-$PWD/rustfs}" +# Where ./install keeps the store: the s3s-fs binary (under bin/), its data +# directory (this entry's "data on local disk"), its pid file, its log and +# the credentials it generated. ./store-start brings it up from here. +export STORE_DIR="${STORE_DIR:-$PWD/store}" # Static keys. ./install generates them once into a file readable by this # user only, and every script reads them from there, so no key appears in a # process argument list. The server and the CLI take RAVEL_S3_ACCESS_KEY and -# RAVEL_S3_SECRET_KEY; the AWS CLI (bucket provisioning and ./data-size) takes -# its own names plus the endpoint. +# RAVEL_S3_SECRET_KEY; s3s-fs takes S3S_FS_ACCESS_KEY and S3S_FS_SECRET_KEY +# (./store-start sets them); the AWS CLI (bucket provisioning and +# ./data-size) takes its own names plus the endpoint. export RAVEL_S3_AUTH=static -if [ -z "${RAVEL_S3_ACCESS_KEY:-}" ] && [ -f "$RUSTFS_DIR/credentials.env" ]; then +if [ -z "${RAVEL_S3_ACCESS_KEY:-}" ] && [ -f "$STORE_DIR/credentials.env" ]; then # shellcheck disable=SC1091 - . "$RUSTFS_DIR/credentials.env" - export RAVEL_S3_ACCESS_KEY="$RUSTFS_ACCESS_KEY" - export RAVEL_S3_SECRET_KEY="$RUSTFS_SECRET_KEY" + . "$STORE_DIR/credentials.env" + export RAVEL_S3_ACCESS_KEY="$STORE_ACCESS_KEY" + export RAVEL_S3_SECRET_KEY="$STORE_SECRET_KEY" fi # The audit-trail tokenization key ./install generated; an unkeyed server # refuses to start without one. -if [ -z "${RAVEL_AUDIT_TOKEN_KEY:-}" ] && [ -f "$RUSTFS_DIR/credentials.env" ]; then - RAVEL_AUDIT_TOKEN_KEY="$(sed -n 's/^RAVEL_AUDIT_TOKEN_KEY=//p' "$RUSTFS_DIR/credentials.env")" +if [ -z "${RAVEL_AUDIT_TOKEN_KEY:-}" ] && [ -f "$STORE_DIR/credentials.env" ]; then + RAVEL_AUDIT_TOKEN_KEY="$(sed -n 's/^RAVEL_AUDIT_TOKEN_KEY=//p' "$STORE_DIR/credentials.env")" fi export RAVEL_AUDIT_TOKEN_KEY="${RAVEL_AUDIT_TOKEN_KEY:-}" unset RAVEL_S3_SESSION_TOKEN AWS_SESSION_TOKEN diff --git a/ravel/rustfs-start b/ravel/rustfs-start deleted file mode 100755 index 5c9ca5fff9..0000000000 --- a/ravel/rustfs-start +++ /dev/null @@ -1,45 +0,0 @@ -#!/bin/bash -# Brings up the local RustFS this entry runs against, if it is not already up. -# Idempotent: ./install calls it after downloading RustFS, and ./start calls it -# before every server start, so a RustFS that died between queries is restarted -# rather than turning the rest of the run into errors. Its data directory is -# on this machine's disk; ./stop leaves RustFS running on purpose, the driver's -# restart cycle is for the database, and the page-cache drop before each cold -# run reaches RustFS's files the same as any local-disk engine's. -set -eu -. ./ravel-env.sh - -# Unauthenticated, so a 200 cannot be confused with a credential problem. -health="${RAVEL_S3_ENDPOINT}/health" -if curl -fsS -o /dev/null "$health" 2>/dev/null; then - exit 0 -fi - -[ -x "$RUSTFS_DIR/rustfs" ] || { echo "rustfs-start: $RUSTFS_DIR/rustfs missing; run ./install first" >&2; exit 1; } -[ -f "$RUSTFS_DIR/credentials.env" ] || { echo "rustfs-start: $RUSTFS_DIR/credentials.env missing; run ./install first" >&2; exit 1; } -mkdir -p "$RUSTFS_DIR/data" - -# Credentials reach RustFS through its environment, never its argument list. -# The console is disabled explicitly: left unset, RustFS still opens its -# console listener on :9001 on every interface. Nothing here uses it, and the -# entry should open exactly two loopback ports, the store's and the database's. -( - set -a - # shellcheck disable=SC1091 - . "$RUSTFS_DIR/credentials.env" - set +a - export RUSTFS_ADDRESS="${RAVEL_S3_ENDPOINT#http://}" - export RUSTFS_CONSOLE_ENABLE=false - nohup "$RUSTFS_DIR/rustfs" server "$RUSTFS_DIR/data" \ - > "$RUSTFS_DIR/rustfs.log" 2>&1 & - echo $! > "$RUSTFS_DIR/rustfs.pid" -) - -for _ in $(seq 1 60); do - if curl -fsS -o /dev/null "$health" 2>/dev/null; then - exit 0 - fi - sleep 1 -done -echo "rustfs-start: RustFS did not answer $health within 60s; see $RUSTFS_DIR/rustfs.log" >&2 -exit 1 diff --git a/ravel/start b/ravel/start index 5946594f2a..e72aee9cd2 100755 --- a/ravel/start +++ b/ravel/start @@ -53,10 +53,10 @@ fi log="server.$(date -u +%Y%m%dT%H%M%SZ).log" ln -sfn "$log" server.log -# The store must be up before the server: a RustFS that died between queries +# The store must be up before the server: an s3s-fs that died between queries # would otherwise turn every later statement into an error. No-op when it is # already serving. -./rustfs-start +./store-start # Each in-flight object-store request holds a socket, and at startup the # server has exited with EMFILE under the common 1024 soft limit. Raise the diff --git a/ravel/store-start b/ravel/store-start new file mode 100755 index 0000000000..d831cf8a2c --- /dev/null +++ b/ravel/store-start @@ -0,0 +1,47 @@ +#!/bin/bash +# Brings up the local s3s-fs this entry runs against, if it is not already up. +# Idempotent: ./install calls it after building s3s-fs, and ./start calls it +# before every server start, so a store that died between queries is restarted +# rather than turning the rest of the run into errors. Its data directory is +# on this machine's disk; ./stop leaves the store running on purpose, the +# driver's restart cycle is for the database, and the page-cache drop before +# each cold run reaches the store's files the same as any local-disk engine's. +set -eu +. ./ravel-env.sh + +# s3s-fs has no health route. Any HTTP answer on the endpoint means it is up; +# the probe is unauthenticated, so an answer cannot be confused with a +# credential problem. +up() { [ "$(curl -s -o /dev/null -w '%{http_code}' "$RAVEL_S3_ENDPOINT/")" != 000 ]; } +if up; then + exit 0 +fi + +[ -x "$STORE_DIR/bin/s3s-fs" ] || { echo "store-start: $STORE_DIR/bin/s3s-fs missing; run ./install first" >&2; exit 1; } +[ -f "$STORE_DIR/credentials.env" ] || { echo "store-start: $STORE_DIR/credentials.env missing; run ./install first" >&2; exit 1; } +mkdir -p "$STORE_DIR/data" + +# Credentials reach s3s-fs through its environment (S3S_FS_ACCESS_KEY and +# S3S_FS_SECRET_KEY), never its argument list. It listens on the loopback +# address and port of RAVEL_S3_ENDPOINT and nothing else. +( + set -a + # shellcheck disable=SC1091 + . "$STORE_DIR/credentials.env" + set +a + export S3S_FS_ACCESS_KEY="$STORE_ACCESS_KEY" + export S3S_FS_SECRET_KEY="$STORE_SECRET_KEY" + hostport="${RAVEL_S3_ENDPOINT#http://}" + nohup "$STORE_DIR/bin/s3s-fs" --host "${hostport%:*}" --port "${hostport##*:}" "$STORE_DIR/data" \ + > "$STORE_DIR/s3s-fs.log" 2>&1 < /dev/null & + echo $! > "$STORE_DIR/s3s-fs.pid" +) + +for _ in $(seq 1 60); do + if up; then + exit 0 + fi + sleep 1 +done +echo "store-start: s3s-fs did not answer $RAVEL_S3_ENDPOINT within 60s; see $STORE_DIR/s3s-fs.log" >&2 +exit 1 From 98c61fff01839619893032afa8ee8a22de50f5aa Mon Sep 17 00:00:00 2001 From: Panagiotis Moustafellos Date: Mon, 26 Oct 2026 04:09:59 +0200 Subject: [PATCH 2/5] ravel: results for v0.26.0 on s3s-fs, c6a.4xlarge One run of the entry as it stands (s3s-fs from the fork release, verified against its SHA256SUMS; the published v0.26.0 binaries verified against the release's SHA256SUMS) on a fresh c6a.4xlarge with this repository's Ubuntu 24.04 image, through the entry's own benchmark.sh. Cold sum 172.8 s against 365.7 s for the same binaries on RustFS (results/20261025); hot 57.0 s, QPS 0.733, error ratio 0.041, 43 of 43 answered, answers equal to the RustFS run's. The RustFS results stay under their own date. --- ravel/results/20261026/c6a.4xlarge.json | 59 +++++++++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100644 ravel/results/20261026/c6a.4xlarge.json diff --git a/ravel/results/20261026/c6a.4xlarge.json b/ravel/results/20261026/c6a.4xlarge.json new file mode 100644 index 0000000000..2d4ee78b4b --- /dev/null +++ b/ravel/results/20261026/c6a.4xlarge.json @@ -0,0 +1,59 @@ +{ + "system": "Ravel", + "date": "2026-10-26", + "machine": "c6a.4xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","S3"], + "load_time": 878, + "data_size": 10206126895, + "concurrent_qps": 0.733, + "concurrent_error_ratio": 0.041, + "result": [ + [0.079, 0.033, 0.033], + [2.575, 0.298, 0.296], + [3.101, 0.433, 0.429], + [3.403, 0.532, 0.522], + [3.601, 1.029, 1.006], + [3.416, 1.343, 1.32], + [0.083, 0.036, 0.035], + [2.566, 0.308, 0.298], + [4.758, 1.368, 1.352], + [5.845, 1.546, 1.514], + [3.951, 0.764, 0.754], + [4.09, 0.836, 0.818], + [3.372, 1.227, 1.213], + [5.39, 1.745, 1.718], + [3.369, 1.334, 1.306], + [3.66, 1.138, 1.112], + [5.579, 2.343, 2.322], + [5.353, 2.218, 2.193], + [5.485, 3.052, 3.078], + [2.919, 0.391, 0.394], + [6.211, 1.688, 1.648], + [7.844, 2.213, 2.184], + [15.047, 3.822, 3.754], + [6.094, 1.742, 1.584], + [0.805, 0.221, 0.287], + [3.199, 0.905, 0.897], + [0.803, 0.256, 0.245], + [6.233, 2.005, 1.982], + [6.727, 4.765, 4.787], + [3.12, 0.427, 0.417], + [5.714, 1.57, 1.534], + [9.108, 1.765, 1.729], + [7.955, 3.594, 3.592], + [7.429, 4.569, 4.539], + [7.424, 4.594, 4.607], + [3.171, 1.059, 1.041], + [0.448, 0.098, 0.095], + [0.454, 0.077, 0.073], + [0.462, 0.084, 0.083], + [0.634, 0.202, 0.193], + [0.428, 0.066, 0.067], + [0.474, 0.066, 0.065], + [0.391, 0.061, 0.055] +] + } From d4189728b05ca897ea18d797edfe77f033f45af5 Mon Sep 17 00:00:00 2001 From: Panagiotis Moustafellos Date: Mon, 26 Oct 2026 15:42:27 +0200 Subject: [PATCH 3/5] ravel: pin the entry to v0.27.0 Install the v0.27.0 release by default, and replace the c6a.4xlarge results file with a run of the entry as it stands on that release (s3s-fs from the fork release, the published v0.27.0 binaries verified against the release's SHA256SUMS): cold sum 163.1 s, hot 51.6 s, concurrent QPS 0.792, error ratio 0.054, 43 of 43 answered. On v0.26.0 the same entry measured 172.8 s cold and 57.0 s hot. The README's resolved-defaults sentence and the cold-sum history carry the v0.27.0 figures. --- ravel/README.md | 10 +-- ravel/install | 2 +- ravel/results/20261026/c6a.4xlarge.json | 94 ++++++++++++------------- 3 files changed, 53 insertions(+), 53 deletions(-) diff --git a/ravel/README.md b/ravel/README.md index 41eedc3564..91d6bf0409 100644 --- a/ravel/README.md +++ b/ravel/README.md @@ -93,8 +93,8 @@ container, less a fixed reserve), with a fixed segment cap and engine deadline. Every resolved value and its source is logged on a `performance default resolved` line in `server.log`; a published result should record those lines, because they are the configuration the numbers were measured at. On -c6a.4xlarge, with the store on loopback, v0.26.0 resolved the fetch cache to -7.0 GB, the per-query SQL pool to 16.5 GB and fetch concurrency to 32. +c6a.4xlarge, with the store on loopback, v0.27.0 resolved the fetch cache to +7.1 GB, the per-query SQL pool to 16.5 GB and fetch concurrency to 32. `./start` also raises the process's open-file soft limit to the hard limit, since the server has exited with `EMFILE` at startup under the common default of 1024. @@ -139,9 +139,9 @@ which reads only the byte ranges a statement needs from each object instead of whole objects; `server.log` names it with `policy_source="default-loopback"`. On c6a.4xlarge, in this repository's fresh-VM setup, with RustFS as the store, the cold sum over the 43 statements was 365.7 s on v0.26.0, against 486.6 s on -v0.23.0 and 1,291.0 s on v0.21.0. The dataset is 10.2 GB. The published -result in `results/` was measured with RustFS; a run of this entry as it -stands, with s3s-fs, replaces it. +v0.23.0 and 1,291.0 s on v0.21.0. With s3s-fs as the store, the entry as it +stands, it was 163.1 s on v0.27.0 (`results/20261026`). The dataset is 10.2 GB. +The RustFS results stay under their own dates. A warm run is served from the read cache or from the store. Since 0.24.0, a server whose store is on loopback gives its fetch cache 24% of its memory diff --git a/ravel/install b/ravel/install index aecba56662..4d81a4e637 100755 --- a/ravel/install +++ b/ravel/install @@ -42,7 +42,7 @@ if ! command -v aws >/dev/null 2>&1; then fi aws --version -RAVEL_VERSION="${RAVEL_VERSION:-v0.26.0}" +RAVEL_VERSION="${RAVEL_VERSION:-v0.27.0}" RAVEL_REPO="${RAVEL_REPO:-https://github.com/NOFireAI/ravel}" case "$(uname -m)" in diff --git a/ravel/results/20261026/c6a.4xlarge.json b/ravel/results/20261026/c6a.4xlarge.json index 2d4ee78b4b..0196759726 100644 --- a/ravel/results/20261026/c6a.4xlarge.json +++ b/ravel/results/20261026/c6a.4xlarge.json @@ -7,53 +7,53 @@ "hardware": "cpu", "tuned": "no", "tags": ["Rust","column-oriented","S3"], - "load_time": 878, - "data_size": 10206126895, - "concurrent_qps": 0.733, - "concurrent_error_ratio": 0.041, + "load_time": 784, + "data_size": 10203840658, + "concurrent_qps": 0.792, + "concurrent_error_ratio": 0.054, "result": [ - [0.079, 0.033, 0.033], - [2.575, 0.298, 0.296], - [3.101, 0.433, 0.429], - [3.403, 0.532, 0.522], - [3.601, 1.029, 1.006], - [3.416, 1.343, 1.32], - [0.083, 0.036, 0.035], - [2.566, 0.308, 0.298], - [4.758, 1.368, 1.352], - [5.845, 1.546, 1.514], - [3.951, 0.764, 0.754], - [4.09, 0.836, 0.818], - [3.372, 1.227, 1.213], - [5.39, 1.745, 1.718], - [3.369, 1.334, 1.306], - [3.66, 1.138, 1.112], - [5.579, 2.343, 2.322], - [5.353, 2.218, 2.193], - [5.485, 3.052, 3.078], - [2.919, 0.391, 0.394], - [6.211, 1.688, 1.648], - [7.844, 2.213, 2.184], - [15.047, 3.822, 3.754], - [6.094, 1.742, 1.584], - [0.805, 0.221, 0.287], - [3.199, 0.905, 0.897], - [0.803, 0.256, 0.245], - [6.233, 2.005, 1.982], - [6.727, 4.765, 4.787], - [3.12, 0.427, 0.417], - [5.714, 1.57, 1.534], - [9.108, 1.765, 1.729], - [7.955, 3.594, 3.592], - [7.429, 4.569, 4.539], - [7.424, 4.594, 4.607], - [3.171, 1.059, 1.041], - [0.448, 0.098, 0.095], - [0.454, 0.077, 0.073], - [0.462, 0.084, 0.083], - [0.634, 0.202, 0.193], - [0.428, 0.066, 0.067], - [0.474, 0.066, 0.065], - [0.391, 0.061, 0.055] + [0.093, 0.034, 0.035], + [1.385, 0.148, 0.141], + [2.117, 0.339, 0.328], + [3.05, 0.471, 0.463], + [3.21, 0.982, 0.958], + [3.084, 1.286, 1.249], + [0.077, 0.034, 0.035], + [1.382, 0.148, 0.141], + [4.418, 1.301, 1.273], + [5.43, 1.438, 1.421], + [3.658, 0.708, 0.7], + [3.778, 0.76, 0.754], + [2.966, 1.13, 1.106], + [5.029, 1.608, 1.582], + [3.034, 1.224, 1.201], + [3.311, 1.087, 1.078], + [5.123, 2.212, 2.224], + [5.063, 2.142, 2.117], + [5.05, 2.907, 2.871], + [2.523, 0.363, 0.357], + [7.37, 1.484, 1.469], + [9.096, 1.772, 1.748], + [16.249, 2.742, 2.729], + [6.774, 1.21, 1.364], + [0.618, 0.168, 0.17], + [2.807, 0.845, 0.828], + [0.584, 0.179, 0.17], + [6.027, 1.97, 1.959], + [6.404, 4.757, 4.786], + [2.184, 0.337, 0.327], + [5.399, 1.435, 1.411], + [8.994, 1.639, 1.603], + [7.863, 3.589, 3.584], + [6.717, 4.03, 4.047], + [6.703, 4.049, 4.06], + [2.827, 0.971, 0.96], + [0.351, 0.099, 0.093], + [0.384, 0.079, 0.079], + [0.363, 0.083, 0.08], + [0.522, 0.183, 0.177], + [0.421, 0.067, 0.068], + [0.391, 0.067, 0.071], + [0.297, 0.06, 0.059] ] } From 866f616c05502670ce5fbd83aa307273967adc5a Mon Sep 17 00:00:00 2001 From: Panagiotis Moustafellos Date: Mon, 26 Oct 2026 15:49:01 +0200 Subject: [PATCH 4/5] ravel: write one-group 46 MB objects on machines with 30 GB or more Add a MemTotal tier at 30 GB to ./load: target 3,700,000,000, one row group per object (--group-target-blocks 512), a 60 s age trigger, pipeline depth 56 and a 14,000,000,000-byte load budget. On c6a.4xlarge with v0.27.0 that writes 223 objects of about 46 MB instead of 422 of about 24 MB, and each column of an object is one contiguous range. Machines below 30 GB keep the existing geometry. RAVEL_GROUP_TARGET_BLOCKS overrides the group setting; an empty value keeps the loader's default. Signed-off-by: Panagiotis Moustafellos --- ravel/README.md | 16 +++++++++++++--- ravel/load | 21 +++++++++++++++++---- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/ravel/README.md b/ravel/README.md index 91d6bf0409..0ae9fb1813 100644 --- a/ravel/README.md +++ b/ravel/README.md @@ -103,7 +103,7 @@ startup under the common default of 1024. in 100,000-row batches. Since 0.23.0 a shard merges several batches into one stored object until its buffer's estimated uncompressed content reaches `--target-bytes`; at 1,850,000,000 that gives objects of about 25 MB on this -dataset (about 410 objects in all; a stored object is about 1/74 of the +dataset (about 420 objects in all on v0.27.0; a stored object is about 1/74 of the estimate here). Before 0.23.0 each batch was its own object, about 4 MB. Large objects are what make a cold scan cheap. The loader holds its batches under `--load-memory-bytes`, so the load fits the machine. `./load` picks the @@ -111,17 +111,27 @@ pipeline depth, read cursors and that budget from `MemTotal`: | MemTotal | `--pipeline-depth` | `--read-cursors` | `--load-memory-bytes` | |---|---|---|---| -| 15 GB and up | 32 | 16 | 6,500,000,000 | +| 30 GB and up | 56 | 16 | 14,000,000,000 | +| 15 to 30 GB | 32 | 16 | 6,500,000,000 | | 7 to 15 GB | 24 | 16 | 3,500,000,000 | | 3 to 7 GB | 16 | 2 | 1,200,000,000 | | under 3 GB | 8 | 1 | derived by the loader | +From 30 GB up `./load` also writes larger objects, each one row group: +`--target-bytes 3700000000` and `--group-target-blocks 512` give objects of +about 46 MB and 450,000 rows (223 objects on c6a.4xlarge with v0.27.0), and +each column of an object is one contiguous range. The age trigger there is +`--max-flush-delay 60s`, since a larger object takes longer to fill. Below +30 GB the target stays at 1,850,000,000 and objects keep the loader's default +row groups. + On a 4 GB machine the budget is smaller than the objects need, so the loader flushes early to stay inside it and the objects come out smaller, about 17 MB median. `--max-flush-delay 30s` lets an object fill before the age trigger closes it. Every value can be overridden from the environment (`RAVEL_BATCH_ROWS`, `RAVEL_TARGET_BYTES`, `RAVEL_PIPELINE_DEPTH`, `RAVEL_READ_CURSORS`, -`RAVEL_LOAD_MEMORY_BYTES`, `RAVEL_MAX_FLUSH_DELAY`). **There is no post-load +`RAVEL_LOAD_MEMORY_BYTES`, `RAVEL_MAX_FLUSH_DELAY`, `RAVEL_GROUP_TARGET_BLOCKS`; +an empty `RAVEL_GROUP_TARGET_BLOCKS` keeps the loader's default). **There is no post-load step**: no compaction, no catalog fold, no VACUUM equivalent, so the layout the queries run against is the layout ingest produced. diff --git a/ravel/load b/ravel/load index 6f906b6fb9..d6e7b2b74c 100755 --- a/ravel/load +++ b/ravel/load @@ -41,8 +41,19 @@ MAPPING="${RAVEL_MAPPING:-hits.mapping.toml}" # budget are sized to the machine's memory below (measured on 128, 16, 8 and # 4 GB hosts, NOFireAI/ravel#2627). Every value can be overridden from the # environment. +# +# From 30 GB up the objects are twice as large and each holds one row group: +# --group-target-blocks 512 keeps a whole object in one group, so each column +# of an object is one contiguous range instead of one per group. At +# 3,700,000,000 that is about 450,000 rows and 46 MB per object, 223 objects +# on c6a.4xlarge. A larger object takes longer to fill, so the age trigger moves to +# 60 s, and the depth and budget grow to hold the batches it needs. mem_kb=$(awk '/^MemTotal:/ {print $2}' /proc/meminfo) -if [ "$mem_kb" -ge 15000000 ]; then +tier_target=1850000000; tier_delay=30s; tier_groups= +if [ "$mem_kb" -ge 30000000 ]; then + tier_depth=56; tier_budget=14000000000; tier_cursors=16 + tier_target=3700000000; tier_delay=60s; tier_groups=512 +elif [ "$mem_kb" -ge 15000000 ]; then tier_depth=32; tier_budget=6500000000; tier_cursors=16 elif [ "$mem_kb" -ge 7000000 ]; then tier_depth=24; tier_budget=3500000000; tier_cursors=16 @@ -54,7 +65,8 @@ else tier_depth=8; tier_budget=; tier_cursors=1 fi load_budget="${RAVEL_LOAD_MEMORY_BYTES-$tier_budget}" -echo "load: MemTotal ${mem_kb} kB, pipeline depth ${RAVEL_PIPELINE_DEPTH:-$tier_depth}, read cursors ${RAVEL_READ_CURSORS:-$tier_cursors}, memory budget ${load_budget:-derived}" >&2 +group_blocks="${RAVEL_GROUP_TARGET_BLOCKS-$tier_groups}" +echo "load: MemTotal ${mem_kb} kB, pipeline depth ${RAVEL_PIPELINE_DEPTH:-$tier_depth}, read cursors ${RAVEL_READ_CURSORS:-$tier_cursors}, memory budget ${load_budget:-derived}, target ${RAVEL_TARGET_BYTES:-$tier_target}, group blocks ${group_blocks:-default}" >&2 "$RAVEL_CLI" --store s3 --s3-auth static load \ --parquet "$PARQUET" \ @@ -65,8 +77,9 @@ echo "load: MemTotal ${mem_kb} kB, pipeline depth ${RAVEL_PIPELINE_DEPTH:-$tier_ --batch-rows "${RAVEL_BATCH_ROWS:-100000}" \ --pipeline-depth "${RAVEL_PIPELINE_DEPTH:-$tier_depth}" \ ${load_budget:+--load-memory-bytes "$load_budget"} \ - --max-flush-delay "${RAVEL_MAX_FLUSH_DELAY:-30s}" \ - --target-bytes "${RAVEL_TARGET_BYTES:-1850000000}" + --max-flush-delay "${RAVEL_MAX_FLUSH_DELAY:-$tier_delay}" \ + --target-bytes "${RAVEL_TARGET_BYTES:-$tier_target}" \ + ${group_blocks:+--group-target-blocks "$group_blocks"} sync From 192956da0076f1d228adf9e6b967c55e3e4e12f6 Mon Sep 17 00:00:00 2001 From: Panagiotis Moustafellos Date: Mon, 26 Oct 2026 16:27:51 +0200 Subject: [PATCH 5/5] ravel: results for the 30 GB load tier on c6a.4xlarge Replace results/20261026/c6a.4xlarge.json with a run of the entry with the 30 GB tier, on the published v0.27.0 binaries and s3s-fs from the fork release: 222 objects, cold sum 117.7 s, hot 51.5 s, concurrent QPS 0.810, error ratio 0.051, load 875 s, 43 of 43 answered. The same release with the previous geometry measured 163.1 s cold and 51.6 s hot. The README's cold-sum sentence carries both figures. Signed-off-by: Panagiotis Moustafellos --- ravel/README.md | 7 +- ravel/results/20261026/c6a.4xlarge.json | 94 ++++++++++++------------- 2 files changed, 51 insertions(+), 50 deletions(-) diff --git a/ravel/README.md b/ravel/README.md index 0ae9fb1813..bd2707b443 100644 --- a/ravel/README.md +++ b/ravel/README.md @@ -149,9 +149,10 @@ which reads only the byte ranges a statement needs from each object instead of whole objects; `server.log` names it with `policy_source="default-loopback"`. On c6a.4xlarge, in this repository's fresh-VM setup, with RustFS as the store, the cold sum over the 43 statements was 365.7 s on v0.26.0, against 486.6 s on -v0.23.0 and 1,291.0 s on v0.21.0. With s3s-fs as the store, the entry as it -stands, it was 163.1 s on v0.27.0 (`results/20261026`). The dataset is 10.2 GB. -The RustFS results stay under their own dates. +v0.23.0 and 1,291.0 s on v0.21.0. With s3s-fs as the store it was 163.1 s on +v0.27.0 with 24 MB objects, and 117.7 s with the 46 MB one-group objects the +30 GB tier writes, the entry as it stands (`results/20261026`). The dataset +is 10.3 GB. The RustFS results stay under their own dates. A warm run is served from the read cache or from the store. Since 0.24.0, a server whose store is on loopback gives its fetch cache 24% of its memory diff --git a/ravel/results/20261026/c6a.4xlarge.json b/ravel/results/20261026/c6a.4xlarge.json index 0196759726..b4f8b16560 100644 --- a/ravel/results/20261026/c6a.4xlarge.json +++ b/ravel/results/20261026/c6a.4xlarge.json @@ -7,53 +7,53 @@ "hardware": "cpu", "tuned": "no", "tags": ["Rust","column-oriented","S3"], - "load_time": 784, - "data_size": 10203840658, - "concurrent_qps": 0.792, - "concurrent_error_ratio": 0.054, + "load_time": 875, + "data_size": 10306343853, + "concurrent_qps": 0.81, + "concurrent_error_ratio": 0.051, "result": [ - [0.093, 0.034, 0.035], - [1.385, 0.148, 0.141], - [2.117, 0.339, 0.328], - [3.05, 0.471, 0.463], - [3.21, 0.982, 0.958], - [3.084, 1.286, 1.249], - [0.077, 0.034, 0.035], - [1.382, 0.148, 0.141], - [4.418, 1.301, 1.273], - [5.43, 1.438, 1.421], - [3.658, 0.708, 0.7], - [3.778, 0.76, 0.754], - [2.966, 1.13, 1.106], - [5.029, 1.608, 1.582], - [3.034, 1.224, 1.201], - [3.311, 1.087, 1.078], - [5.123, 2.212, 2.224], - [5.063, 2.142, 2.117], - [5.05, 2.907, 2.871], - [2.523, 0.363, 0.357], - [7.37, 1.484, 1.469], - [9.096, 1.772, 1.748], - [16.249, 2.742, 2.729], - [6.774, 1.21, 1.364], - [0.618, 0.168, 0.17], - [2.807, 0.845, 0.828], - [0.584, 0.179, 0.17], - [6.027, 1.97, 1.959], - [6.404, 4.757, 4.786], - [2.184, 0.337, 0.327], - [5.399, 1.435, 1.411], - [8.994, 1.639, 1.603], - [7.863, 3.589, 3.584], - [6.717, 4.03, 4.047], - [6.703, 4.049, 4.06], - [2.827, 0.971, 0.96], - [0.351, 0.099, 0.093], - [0.384, 0.079, 0.079], - [0.363, 0.083, 0.08], - [0.522, 0.183, 0.177], - [0.421, 0.067, 0.068], - [0.391, 0.067, 0.071], - [0.297, 0.06, 0.059] + [0.087, 0.026, 0.026], + [0.728, 0.138, 0.137], + [1.001, 0.328, 0.328], + [1.348, 0.465, 0.456], + [1.605, 0.964, 0.958], + [1.777, 1.233, 1.238], + [0.078, 0.027, 0.026], + [0.734, 0.143, 0.137], + [2.291, 1.287, 1.284], + [3.251, 1.436, 1.413], + [1.664, 0.7, 0.687], + [1.785, 0.743, 0.745], + [1.64, 1.125, 1.109], + [2.809, 1.604, 1.592], + [1.771, 1.211, 1.198], + [1.711, 1.074, 1.063], + [3.04, 2.217, 2.207], + [2.935, 2.129, 2.124], + [3.577, 2.825, 2.888], + [1.288, 0.365, 0.381], + [6.744, 1.499, 1.485], + [7.864, 1.789, 1.77], + [13.696, 2.81, 2.765], + [6.518, 1.319, 1.092], + [0.447, 0.167, 0.205], + [1.292, 0.823, 0.83], + [0.44, 0.169, 0.146], + [5.225, 1.962, 1.949], + [5.979, 4.69, 4.763], + [1.064, 0.315, 0.314], + [3.804, 1.432, 1.406], + [6.984, 1.652, 1.642], + [5.864, 3.565, 3.574], + [5.898, 4.073, 4.072], + [6.217, 4.093, 4.069], + [1.929, 0.983, 0.99], + [0.374, 0.1, 0.094], + [0.325, 0.076, 0.075], + [0.322, 0.087, 0.086], + [0.549, 0.175, 0.176], + [0.417, 0.063, 0.058], + [0.342, 0.065, 0.063], + [0.279, 0.054, 0.053] ] }