Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,15 @@ INSERT, COPY, ATTACH, PRAGMA and friends outright, so the rule is the parser's,
not a keyword denylist. Filesystem and network access are disabled before any
project SQL executes, models included: a forked recipe is a stranger's code.

The session runs in UTC. rindexer exports `block_timestamp` as `TIMESTAMP WITH
TIME ZONE`, and DuckDB renders, casts and buckets that type in the session
`TimeZone`, which otherwise follows the machine. The worker pins it, so
`strftime`, `date_trunc`, `hour()` and casts to `TIMESTAMP` give the same
answer on the producer, in a fork in another zone, and in `query` on a laptop.
This is a worker-wide contract, not a per-query option; it needs DuckDB's ICU
extension, which the bundled binaries link statically, and the worker refuses
to run project SQL on a build without it.

`cp_sortkey(v)` is available in every query. It maps a decimal-string amount to
a fixed-width key whose lexicographic order is signed-numeric order, so
`ORDER BY cp_sortkey(value)` sorts uint256 correctly without projecting a
Expand Down
2 changes: 1 addition & 1 deletion examples/usdc-supply/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ RPC_URL=
# nothing to install or point at yourself.
DATABASE_URL=postgresql://chainplot:chainplot@postgres:5432/chainplot

# Producer image tag. Build it once from a chainplot checkout:
# Producer image tag. Build it once from a Chainplot checkout:
# docker build --platform linux/amd64 -t chainplot:local \
# -f docker/producer.Dockerfile .
# CHAINPLOT_IMAGE=chainplot:local
2 changes: 1 addition & 1 deletion examples/usdc-supply/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ indexed filters, a model, and time-series charts.

## Run it

Build the producer image once from a chainplot checkout, then:
Build the producer image once from a Chainplot checkout, then:

```bash
cp .env.example .env # fill in RPC_URL
Expand Down
6 changes: 3 additions & 3 deletions examples/usdc-supply/compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ services:
retries: 30

producer:
# The producer image is the chainplot CLI plus the pinned rindexer binary.
# It is built from the chainplot repo, not from this project — a scaffolded
# project has no CLI sources to build from. Once, from a chainplot checkout:
# The producer image is the Chainplot CLI plus the pinned rindexer binary.
# It is built from the Chainplot repo, not from this project — a scaffolded
# project has no CLI sources to build from. Once, from a Chainplot checkout:
#
# docker build --platform linux/amd64 \
# -t chainplot:local -f docker/producer.Dockerfile .
Expand Down
18 changes: 18 additions & 0 deletions src/query/workerMain.ts
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,24 @@ async function execute(req: WorkerRequest): Promise<{
// trusted than the query itself, so they must land on this side of it.
// DuckDB does not allow re-enabling external access in a session.
await conn.run("SET enable_external_access=false");
// rindexer exports block_timestamp as TIMESTAMP WITH TIME ZONE, and
// DuckDB renders, casts and buckets that type in the session's
// TimeZone, which defaults to the machine's. Pinning UTC is what makes
// an hourly or daily figure the same on the producer, in a fork on a
// laptop in another zone, and in the published results. The setting
// belongs to DuckDB's ICU extension, which the bundled binaries link
// statically; a build without it must fail here, before any project
// SQL, rather than quietly compute in local time.
try {
await conn.run("SET TimeZone='UTC'");
} catch (err) {
throw issueError({
code: "internal",
message:
"DuckDB build lacks ICU time-zone support; the query worker requires it " +
`so timestamps compute in UTC on every machine (${err instanceof Error ? err.message : String(err)})`,
});
}
await conn.run(SORT_KEY_MACRO);

for (const model of req.models ?? []) {
Expand Down
2 changes: 1 addition & 1 deletion templates/ingest-transfers/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ RPC_URL=
# nothing to install or point at yourself.
DATABASE_URL=postgresql://chainplot:chainplot@postgres:5432/chainplot

# Producer image tag. Build it once from a chainplot checkout:
# Producer image tag. Build it once from a Chainplot checkout:
# docker build --platform linux/amd64 -t chainplot:local \
# -f docker/producer.Dockerfile .
# CHAINPLOT_IMAGE=chainplot:local
4 changes: 2 additions & 2 deletions templates/ingest-transfers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ thing you supply is an archive-capable RPC endpoint.

## One-time: build the producer image

The producer image is the chainplot CLI plus rindexer. It is built from a
chainplot checkout, not from this project — a scaffolded project has no CLI
The producer image is the Chainplot CLI plus rindexer. It is built from a
Chainplot checkout, not from this project — a scaffolded project has no CLI
sources. rindexer ships linux/amd64 only, so build for that platform:

```bash
Expand Down
6 changes: 3 additions & 3 deletions templates/ingest-transfers/compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ services:
retries: 30

producer:
# The producer image is the chainplot CLI plus the pinned rindexer binary.
# It is built from the chainplot repo, not from this project — a scaffolded
# project has no CLI sources to build from. Once, from a chainplot checkout:
# The producer image is the Chainplot CLI plus the pinned rindexer binary.
# It is built from the Chainplot repo, not from this project — a scaffolded
# project has no CLI sources to build from. Once, from a Chainplot checkout:
#
# docker build --platform linux/amd64 \
# -t chainplot:local -f docker/producer.Dockerfile .
Expand Down
67 changes: 67 additions & 0 deletions tests/query/timezone.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import { describe, expect, it } from "vitest";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { DuckDBInstance } from "@duckdb/node-api";
import { runQuery } from "../../src/query/runQuery.js";

// A snapshot shaped like rindexer's export: block_timestamp is TIMESTAMP WITH
// TIME ZONE. Written by this test process, not the worker, which has external
// access switched off before any project SQL runs.
async function timestampFixture(): Promise<string> {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "chainplot-tz-"));
const parquet = path.join(dir, "events.parquet");
const instance = await DuckDBInstance.create(":memory:");
const conn = await instance.connect();
await conn.run(
`COPY (
SELECT * FROM (VALUES
(1, TIMESTAMPTZ '2026-09-16 00:26:20+00'),
(2, TIMESTAMPTZ '2026-09-16 23:59:59+00')
) AS t(rindexer_id, block_timestamp)
) TO '${parquet.replaceAll("'", "''")}' (FORMAT PARQUET)`,
);
return parquet;
}

// DuckDB formats, casts and buckets TIMESTAMPTZ in the session TimeZone, which
// follows the machine unless pinned: the same query would publish "08:00" from
// a laptop in Singapore and "00:00" from the producer container. The worker
// pins UTC so a fork reproduces the published figures wherever it runs.
describe("query session timezone", () => {
// TimeZone is a setting DuckDB's ICU extension registers, so reading it
// back also proves the bundled build links ICU: without it the worker's
// SET would have failed before this query ran. (duckdb_extensions() is not
// usable here: it scans the extension directory, which the worker's
// external-access lockdown forbids.)
it("is pinned to UTC by the ICU-backed setting", async () => {
const parquet = await timestampFixture();
const result = await runQuery({
sql: "SELECT current_setting('TimeZone') AS tz FROM (SELECT 1)",
tables: { events: parquet },
rawAmountColumns: [],
rowLimit: 10,
});
expect(result.rows).toEqual([["UTC"]]);
});

it("renders, buckets and casts a TIMESTAMPTZ column in UTC", async () => {
const parquet = await timestampFixture();
const result = await runQuery({
sql: `SELECT typeof(block_timestamp) AS t,
strftime(block_timestamp, '%Y-%m-%d %H:%M:%S') AS rendered,
strftime(date_trunc('day', block_timestamp), '%Y-%m-%d') AS day,
hour(block_timestamp) AS h,
strftime(block_timestamp::TIMESTAMP, '%H:%M') AS cast_naive
FROM events ORDER BY rindexer_id`,
tables: { events: parquet },
rawAmountColumns: [],
rowLimit: 10,
});
expect(result.rows).toEqual([
// hour() is BIGINT, which the worker serializes as a decimal string.
["TIMESTAMP WITH TIME ZONE", "2026-09-16 00:26:20", "2026-09-16", "0", "00:26"],
["TIMESTAMP WITH TIME ZONE", "2026-09-16 23:59:59", "2026-09-16", "23", "23:59"],
]);
});
});
88 changes: 83 additions & 5 deletions tests/viewer/format.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import {
columnLabel,
compareValues,
displayAmount,
isNumericColumn,
formatCell,
groupDigits,
relativeTime,
Expand Down Expand Up @@ -214,20 +215,35 @@ describe("display rounding", () => {
});

it("rounds half away from zero rather than truncating", () => {
expect(displayAmount(1_999_999n, 6)).toBe("2");
expect(displayAmount(-1_999_999n, 6)).toBe("-2");
expect(displayAmount(1_999_999n, 6)).toBe("2.00");
expect(displayAmount(-1_999_999n, 6)).toBe("-2.00");
expect(displayAmount(1_005_000n, 6)).toBe("1.01");
});

// A column of amounts is read down the decimal point. Trimming the zeros
// here would print "9,000,000", "1,499,999.5" and "101,570,558.71" in one
// column, which is what the exact rendering is for.
it("pads every rounded figure to the same width", () => {
expect(displayAmount(9_000_000_000_000n, 6)).toBe("9,000,000.00");
expect(displayAmount(1_499_999_500_000n, 6)).toBe("1,499,999.50");
expect(displayAmount(101_570_558_713_200n, 6)).toBe("101,570,558.71");
});

it("still trims trailing zeros when rendering an exact value", () => {
expect(scaleAmount(9_000_000_000_000n, 6)).toBe("9,000,000");
expect(scaleAmount(1_499_999_500_000n, 6)).toBe("1,499,999.5");
});

it("keeps full precision below one unit, where the fraction is the value", () => {
expect(displayAmount(1n, 18)).toBe("0.000000000000000001");
expect(displayAmount(-1n, 18)).toBe("-0.000000000000000001");
});

it("leaves a value that needs no rounding untouched", () => {
it("pads a value that needs no rounding, and reveals it exactly on hover", () => {
const cell = formatCell("1500000", usdc);
expect(cell.text).toBe("1.5 USDC");
expect(cell.exact).toBe("1500000");
expect(cell.text).toBe("1.50 USDC");
expect(cell.exact).toContain("1.5 USDC");
expect(cell.exact).toContain("1500000");
});

it("does not round integer columns that are not amounts", () => {
Expand Down Expand Up @@ -281,3 +297,65 @@ describe("rowWindow", () => {
expect(rowWindow(5000, 0, 400, 0, opts).virtual).toBe(false);
});
});

// Alignment follows the column, not the value. Deciding per cell put a null,
// which has no digits, out of line with the figures above it, and left-aligned
// a header over right-aligned cells.
describe("isNumericColumn", () => {
it("treats a declared amount as numeric however it is typed", () => {
expect(
isNumericColumn({ name: "v", logical_type: "VARCHAR", raw_amount: true }, [null]),
).toBe(true);
});

it("treats a numeric logical type as numeric even when every value is null", () => {
expect(isNumericColumn({ name: "n", logical_type: "BIGINT" }, [null, null])).toBe(true);
expect(isNumericColumn({ name: "d", logical_type: "DECIMAL(18,3)" }, [])).toBe(true);
expect(isNumericColumn({ name: "h", logical_type: "HUGEINT" }, [])).toBe(true);
expect(isNumericColumn({ name: "u", logical_type: "UBIGINT" }, [])).toBe(true);
});

it("settles a VARCHAR by its values, so an undeclared uint256 still reads as one", () => {
const column = { name: "v", logical_type: "VARCHAR" };
expect(isNumericColumn(column, ["1", "115792089237316195423570985008687907853269984665640564039457584007913129639935"])).toBe(true);
expect(isNumericColumn(column, ["1", null, "2"])).toBe(true);
});

// A list of numbers is not a number: it renders as "[1, 2, 3]" and belongs
// on the left with the text. An unanchored type match accepts "INTEGER[]".
it("does not treat a list of numbers as a number column", () => {
for (const logical_type of ["INTEGER[]", "UBIGINT[]", "DECIMAL(18,3)[]", "DOUBLE[]"]) {
expect(isNumericColumn({ name: "l", logical_type }, [])).toBe(false);
expect(isNumericColumn({ name: "l", logical_type }, [[1, 2]])).toBe(false);
}
expect(isNumericColumn({ name: "s", logical_type: "STRUCT(a INTEGER)" }, [])).toBe(false);
});

it("accepts the scalar numeric types as DuckDB spells them", () => {
for (const logical_type of [
"TINYINT", "SMALLINT", "INTEGER", "BIGINT", "HUGEINT",
"UTINYINT", "USMALLINT", "UINTEGER", "UBIGINT", "UHUGEINT",
"FLOAT", "REAL", "DOUBLE", "DECIMAL(18,3)", "DECIMAL",
]) {
expect(isNumericColumn({ name: "n", logical_type }, [])).toBe(true);
}
});

// Only text is settled by its values. A BIT string is "0101" and an enum
// label can be any text at all; neither is a number because it reads like
// digits.
it("does not infer from the values of a type that is not text", () => {
expect(isNumericColumn({ name: "b", logical_type: "BIT" }, ["0101", "1"])).toBe(false);
expect(isNumericColumn({ name: "e", logical_type: "ENUM('1','2')" }, ["1", "2"])).toBe(false);
expect(isNumericColumn({ name: "d", logical_type: "DATE" }, ["20260916"])).toBe(false);
expect(isNumericColumn({ name: "u", logical_type: "UUID" }, ["12345"])).toBe(false);
});

it("does not call a VARCHAR numeric on the strength of some of its values", () => {
const column = { name: "v", logical_type: "VARCHAR" };
expect(isNumericColumn(column, ["1", "Ethereum"])).toBe(false);
expect(isNumericColumn(column, ["00:26:20"])).toBe(false);
expect(isNumericColumn(column, [])).toBe(false);
expect(isNumericColumn(column, [null, null])).toBe(false);
});
});
34 changes: 30 additions & 4 deletions viewer/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,15 @@ import {
columnLabel,
compareValues,
formatCell,
isNumericColumn,
relativeTime,
rowWindow,
toChartNumber,
} from "./format.js";

/** Where a reader learns how to rebuild a release from its dataset. */
const FORK_HOWTO_URL = "https://github.com/chainstacklabs/chainplot#fork-a-published-release";
const CHAINPLOT_REPO_URL = "https://github.com/chainstacklabs/chainplot";
const FORK_HOWTO_URL = `${CHAINPLOT_REPO_URL}#fork-a-published-release`;

/** Columns the panel asked to compute but not show, e.g. an explicit sort key. */
function visibleColumns(
Expand Down Expand Up @@ -160,6 +162,20 @@ function DataTable({
const [rowHeight, setRowHeight] = useState(ASSUMED_ROW_HEIGHT);
const bodyRef = useRef<HTMLTableSectionElement | null>(null);

// Alignment is decided once per column, so a header, its figures and a null
// among them all sit on the same edge.
const numericColumns = useMemo(() => {
const numeric = new Set<number>();
for (const index of columns) {
const column = result.columns[index];
if (!column) continue;
if (isNumericColumn(column, result.rows.map((row) => row[index]))) {
numeric.add(index);
}
}
return numeric;
}, [columns, result]);

const sorted = useMemo(() => {
if (sortCol === null) return result.rows;
const rows = [...result.rows];
Expand Down Expand Up @@ -220,7 +236,13 @@ function DataTable({
<th
key={column.name}
aria-sort={active ? (asc ? "ascending" : "descending") : "none"}
className={column.raw_amount ? "numeric raw" : "text"}
className={
numericColumns.has(index)
? column.raw_amount
? "numeric raw"
: "numeric"
: "text"
}
>
<button
type="button"
Expand Down Expand Up @@ -256,7 +278,7 @@ function DataTable({
return (
<td
key={column.name}
className={cell.numeric ? "numeric" : "text"}
className={numericColumns.has(index) ? "numeric" : "text"}
title={cell.text === cell.exact ? undefined : cell.exact}
>
{cell.text}
Expand Down Expand Up @@ -528,7 +550,11 @@ export function App() {
))}

<footer className="colophon">
Built by <span>chainplot</span> · {release.mode.replace(/_/g, " ")} ·{" "}
Built by{" "}
<a className="brand" href={CHAINPLOT_REPO_URL} rel="noopener">
Chainplot
</a>{" "}
· {release.mode.replace(/_/g, " ")} ·{" "}
<time dateTime={release.generated_at}>{release.generated_at}</time>
{" · "}
{release.mode === "results_only" ? (
Expand Down
Loading
Loading