Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions docs/account-facts-holdings-rollout.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Read-only holdings: staged rollout and rollback

This runbook describes the release gates for the optional account-facts holdings
pair. It does not authorize a merge, deployment, publisher change, live write,
forced broker run, or trade. PR #617 stays draft pending owner review. Merging
console changes to `main` triggers `deploy-strategy-switch-console.yml` and can
change the live console, so release requires separate owner approval.

## Release order

1. **Receiver first.** Review and release the compatible receiver/frontend while
both publishers keep stripping the two archive-only fields. Existing payloads
remain valid; a missing pair displays “No holdings data yet”. Do not block this
receiver stage on publishers sending holdings: the old exact-key receiver
rejects the extended payload. Record the deployed receiver revision and verify
ordinary read-only account-facts health before enabling any publisher change.
2. **Runtime evidence second.** Verify the approved IBKR/Schwab runtime revisions
and a normally produced run report in restricted evidence storage. Require
`broker_reported_positions` and `broker_reported_positions_scope` together,
with IBKR `stocks_only` or Schwab `strategy_symbols_only`. Verify valid decimal
strings, unique uppercase symbols, the supported row fields and 1–64 rows.
A runtime release or synthetic test alone does not prove an actual report has
this contract. Missing, empty, invalid or unscoped data is not zero holdings.
Keep publisher stripping enabled while evidence is incomplete.
3. **Publisher last, separately approved.** Only after both receiver adoption and
runtime evidence are verified, review a separate change to IBKR
`publish_account_facts_from_report.py` and Schwab
`publish_account_facts_from_reports.py`. Preserve paired-field validation and
existing authentication/source bindings. Verify normal publish acceptance and
authorized read-only readback, including the displayed scope and freshness.
Never broaden partial coverage into a full-account inventory claim.

Keep actual reports, holdings, balances, native account identifiers and credentials
out of public PRs, fixtures and CI logs. Publish only synthetic examples and
non-sensitive revision/check outcomes. These release gates are not live evidence.

## Retention and rollback

- Holdings are part of the persisted latest **and daily** history payloads. The
current history policy retains at most 366 UTC observation days; UI staleness
only hides holdings and does not delete those stored payloads.
- First restore publisher stripping (both fields together) if publication needs
to roll back. A newer ordinary payload without positions replaces the latest
holdings instead of carrying them forward. It does not erase older daily rows.
- Do not simply redeploy the pre-holdings receiver after accepting extended
payloads. Its exact-key validation rejects them on ingress and when projecting
persisted history, potentially making balances/history unavailable as well.
Prefer retaining the compatible receiver and disabling publication/display.
Any downgrade needs a separately reviewed compatibility or data-migration plan;
stopping publishers or waiting for UI staleness is not such a plan. Do not
delete retained evidence to force compatibility without explicit approval.
- If another console PR lands first, reconcile this branch with the new base and
regenerate `v2_asset_map.js`; do not hand-merge the generated payload.

## Reproducible checks (synthetic/offline)

```sh
npm ci --prefix web/strategy-switch-console
npm ci --prefix web/strategy-switch-console/frontend
npm run build --prefix web/strategy-switch-console/frontend
npm run sync:v2-assets --prefix web/strategy-switch-console/frontend
git diff --exit-code -- web/strategy-switch-console/v2_asset_map.js
CLOUDFLARE_CF_FETCH_ENABLED=false node tests/account_facts_validation.mjs
node --experimental-strip-types tests/account_history_presentation_validation.mjs
git diff --check
```

The receiver suite covers legacy payloads, paired scope validation, row bounds,
stale suppression, durable readback/idempotency, rejected-write preservation,
newer position-less snapshots and retained previous-day holdings. The existing
repository CI remains required; synthetic checks do not prove production adoption.

## 中文:发布与回退边界

本说明不授权合并、部署、修改发布器、生产写入、强制券商运行或交易。#617
保持草稿,等待所有者审阅;控制台改动合入 `main` 会触发自动部署,需单独批准。

1. 先发布兼容接收端与前端,两个发布器继续剥离持仓字段。旧载荷仍可接收,缺少
持仓时显示“暂无持仓数据”。旧接收端拒绝扩展字段,不能把发布器先发送持仓
作为接收端发布的前置条件。记录接收端版本并核验普通只读健康状态。
2. 再核验获批的运行时版本及正常运行产生的受限报告:持仓与范围必须成对,IBKR
为 `stocks_only`,Schwab 为 `strategy_symbols_only`,且满足字段、十进制字符串、
大写唯一标的及 1–64 行限制。版本发布或合成测试不等于真实报告证据;缺失、空值
或格式错误不代表零持仓。证据未齐前继续剥离。
3. 最后在另行批准的 PR 中修改两个发布器,核验正常发送及授权只读回读的范围与
新鲜度。不得把部分覆盖解释为账户全部持仓。实际报告、持仓、余额、账户标识与
凭据不得进入公开 PR、测试夹具或 CI 日志。

持仓会进入最新快照和每日历史;历史最多保留 366 个 UTC 观察日。界面过期隐藏
不等于删除。回退应先恢复发布器成对剥离;更新的无持仓快照会清除最新展示,但
不会清除旧日期的持仓。旧接收端也会拒绝读取已持久化的扩展载荷,影响余额及历史
可用性,因此不能直接降级。优先保留兼容接收端并停用持仓发送/展示;降级需另行
审阅兼容或迁移方案,不能靠等待过期或未经批准删除证据解决。若其他控制台 PR
先合入,先协调基线,再按上述构建脚本重新生成资产文件,不手工合并生成内容。
上述测试覆盖兼容、范围、边界、持久化、拒写保护及历史保留,不证明生产已采用。
110 changes: 110 additions & 0 deletions tests/account_facts_validation.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import {
ACCOUNT_FACTS_SOURCE_KIND,
ACCOUNT_FACTS_STALE_MS,
ACCOUNT_FACTS_PLATFORMS,
ACCOUNT_FACTS_POSITIONS_SCOPES,
IBKR_ACCOUNT_FACTS_HISTORY_SCHEMA,
IBKR_ACCOUNT_FACTS_SNAPSHOT_SCHEMA,
IBKR_ACCOUNT_FACTS_SOURCE_KIND,
Expand Down Expand Up @@ -1213,6 +1214,76 @@ const normalizedIbkrHistory = normalizeAccountFactsHistoryPayload(acceptedIbkrHi
});
assert.equal(normalizedIbkrHistory.broker_reported_balances[0].net_assets, null);
assert.deepEqual(normalizedIbkrHistory.account_ids, ["U16608560"]);

// Optional broker-reported positions (read-only holdings panel).
assert.equal(Object.hasOwn(normalizedIbkrHistory, "broker_reported_positions"), false, "history without positions stays readable");
const ibkrPositions = [
{ symbol: "SOXL", quantity: "4", market_value: "635.92", currency: "USD" },
{ symbol: "BRK.B", quantity: "1.5", market_value: "700", currency: "USD", avg_cost: "450.1" },
];
const ibkrWithPositions = normalizeAccountFactsHistoryPayload(ibkrHistory({
broker_reported_positions: ibkrPositions,
broker_reported_positions_scope: "stocks_only",
}));
assert.deepEqual(ibkrWithPositions.broker_reported_positions, ibkrPositions);
assert.equal(ibkrWithPositions.broker_reported_positions_scope, ACCOUNT_FACTS_POSITIONS_SCOPES.ibkr);
for (const patch of [
{ broker_reported_positions: ibkrPositions },
{ broker_reported_positions_scope: "stocks_only" },
{ broker_reported_positions: ibkrPositions, broker_reported_positions_scope: "strategy_symbols_only" },
{ broker_reported_positions: [], broker_reported_positions_scope: "stocks_only" },
{ broker_reported_positions: [ibkrPositions[0], ibkrPositions[0]], broker_reported_positions_scope: "stocks_only" },
{ broker_reported_positions: [{ ...ibkrPositions[0], quantity: 4 }], broker_reported_positions_scope: "stocks_only" },
{ broker_reported_positions: [{ ...ibkrPositions[0], symbol: "soxl" }], broker_reported_positions_scope: "stocks_only" },
{ broker_reported_positions: [{ ...ibkrPositions[0], currency: "BASE" }], broker_reported_positions_scope: "stocks_only" },
{ broker_reported_positions: [{ ...ibkrPositions[0], account_id: "U1" }], broker_reported_positions_scope: "stocks_only" },
{ broker_reported_positions: Array.from({ length: 65 }, (_, i) => ({ ...ibkrPositions[0], symbol: `S${i}` })), broker_reported_positions_scope: "stocks_only" },
]) {
assert.throws(() => normalizeAccountFactsHistoryPayload(ibkrHistory(patch)), /invalid_account_facts_(positions|history)/);
}
// Both optional fields are additive; other platforms and malformed pairs stay closed.
for (const makeHistory of [ibkrHistory, schwabHistory]) {
const scope = makeHistory === ibkrHistory ? "stocks_only" : "strategy_symbols_only";
assert.doesNotThrow(() => normalizeAccountFactsHistoryPayload(makeHistory()));
for (const rows of [null, {}, [null], [{ ...ibkrPositions[0], avg_cost: 1 }],
[{ ...ibkrPositions[0], quantity: "1e3" }], [{ ...ibkrPositions[0], market_value: "NaN" }]]) {
assert.throws(() => normalizeAccountFactsHistoryPayload(makeHistory({
broker_reported_positions: rows, broker_reported_positions_scope: scope,
})), /invalid_account_facts_positions/);
}
const maximumRows = Array.from({ length: 64 }, (_, i) => ({ ...ibkrPositions[0], symbol: `S${i}` }));
assert.equal(normalizeAccountFactsHistoryPayload(makeHistory({
broker_reported_positions: maximumRows, broker_reported_positions_scope: scope,
})).broker_reported_positions.length, 64);
}
assert.throws(() => normalizeAccountFactsHistoryPayload(historyPayload({
broker_reported_positions: ibkrPositions, broker_reported_positions_scope: "stocks_only",
})), /invalid_account_facts_positions/);

const ibkrPositionsReadModel = buildAccountFactsReadModel({
accountOptions: { ibkr: [ibkrAccount] },
bindings: { schema_version: ACCOUNT_FACTS_BINDINGS_SCHEMA, bindings: [ibkrBinding] },
storedByAccount: new Map([[`ibkr:${ibkrAccount.key}`, ibkrWithPositions]]),
now: Date.parse(ibkrWithPositions.observed_finished_at) + 1000,
});
assert.deepEqual(ibkrPositionsReadModel.accounts[0].positions, ibkrPositions);
assert.equal(ibkrPositionsReadModel.accounts[0].positions_scope, "stocks_only");
const staleIbkrPositionsReadModel = buildAccountFactsReadModel({
accountOptions: { ibkr: [ibkrAccount] },
bindings: { schema_version: ACCOUNT_FACTS_BINDINGS_SCHEMA, bindings: [ibkrBinding] },
storedByAccount: new Map([[`ibkr:${ibkrAccount.key}`, ibkrWithPositions]]),
now: Date.parse(ibkrWithPositions.observed_finished_at) + ACCOUNT_FACTS_STALE_MS + 1,
});
assert.equal(Object.hasOwn(staleIbkrPositionsReadModel.accounts[0], "positions"), false, "stale positions are not shown");
const schwabWithPositions = normalizeAccountFactsHistoryPayload(schwabHistory({
broker_reported_positions: [{ symbol: "TQQQ", quantity: "10", market_value: "900", currency: "USD" }],
broker_reported_positions_scope: "strategy_symbols_only",
}));
assert.equal(schwabWithPositions.broker_reported_positions_scope, "strategy_symbols_only");
assert.throws(() => normalizeAccountFactsHistoryPayload(schwabHistory({
broker_reported_positions: [{ symbol: "TQQQ", quantity: "10", market_value: "900", currency: "USD" }],
broker_reported_positions_scope: "stocks_only",
})), /invalid_account_facts_positions/);
assert.throws(() => normalizeAccountFactsHistoryPayload(ibkrHistory({
broker_account_type: { value: "MARGIN", source_tag: "securitiesAccount.type" },
})), /invalid_account_facts_history/);
Expand Down Expand Up @@ -1295,6 +1366,40 @@ const ibkrHkdHistory = await (await getHistory("ibkr", ibkrAccount.key, "HKD", s
assert.equal(ibkrHkdHistory.series.points[0].net_assets, "99.25");
assert.equal(ibkrHkdHistory.series.points[0].total_cash, null);

// Receiver-first rollout: old publishers work, paired fields survive storage/readback,
// invalid updates cannot overwrite them, and publisher rollback removes latest holdings.
const holdingsStarted = new Date(Date.now() - 50_000).toISOString();
const holdingsFinished = new Date(Date.now() - 40_000).toISOString();
const holdingsHistory = ibkrHistory({
observed_started_at: holdingsStarted, observed_finished_at: holdingsFinished,
broker_reported_positions: ibkrPositions, broker_reported_positions_scope: "stocks_only",
});
const holdingsWrite = await post(holdingsHistory, ibkrAuth, env);
assert.equal(holdingsWrite.status, 200);
const holdingsAck = await holdingsWrite.json();
assert.equal(Object.hasOwn(holdingsAck, "broker_reported_positions"), false);
const readIbkrHoldings = async () => (await (await get(sessionHeaders, env)).json()).accounts
.find(item => item.platform === "ibkr" && item.account_key === ibkrAccount.key);
assert.deepEqual((await readIbkrHoldings()).positions, ibkrPositions);
assert.equal((await readIbkrHoldings()).positions_scope, "stocks_only");
assert.equal((await (await post(holdingsHistory, ibkrAuth, env)).json()).unchanged, true);
for (const patch of [
{ broker_reported_positions_scope: "strategy_symbols_only" },
{ broker_reported_positions: [] },
]) {
assert.equal((await post({ ...holdingsHistory, ...patch }, ibkrAuth, env)).status, 400);
assert.deepEqual((await readIbkrHoldings()).positions, ibkrPositions, "rejected updates preserve stored holdings");
}
const publisherRollback = await post(ibkrHistory({
observed_started_at: new Date(Date.now() - 35_000).toISOString(),
observed_finished_at: new Date(Date.now() - 30_000).toISOString(),
}), ibkrAuth, env);
assert.equal(publisherRollback.status, 200);
const rollbackRead = await readIbkrHoldings();
assert.equal(rollbackRead.data_status, "fresh");
assert.equal(Object.hasOwn(rollbackRead, "positions"), false, "missing positions never carry forward from an older snapshot");
assert.equal(Object.hasOwn(rollbackRead, "positions_scope"), false);

// A server-approved source revision may rotate while the account identity stays fixed.
// Old data must disappear under the new binding until a new-source observation arrives.
const rotatedIbkrBinding = {
Expand Down Expand Up @@ -1414,6 +1519,8 @@ const rotationIbkrOldHistory = {
target_id: rotationIbkrOldBinding.target_id,
source_binding: { kind: IBKR_ACCOUNT_FACTS_SOURCE_KIND, status: "bound", id: rotationIbkrOldId },
observation_date: observationDateFrom(previousDayStartedAt),
broker_reported_positions: ibkrPositions,
broker_reported_positions_scope: "stocks_only",
broker_reported_balances: [{ currency: "USD", net_assets: "222" }],
cash: [{ currency: "USD", cash_balance: "2", source_tag: "CashBalance" }],
};
Expand Down Expand Up @@ -1505,6 +1612,9 @@ const retainedIbkrOld = await runtimeCommand({
source_binding_id: rotationIbkrOldId,
});
assert.deepEqual(retainedIbkrOld.days.map((day) => day.payload.broker_reported_balances[0].net_assets), ["222"]);
assert.deepEqual(retainedIbkrOld.days[0].payload.broker_reported_positions, ibkrPositions,
"old daily payloads retain holdings even after a source change and a newer position-less snapshot");
assert.equal(retainedIbkrOld.days[0].payload.broker_reported_positions_scope, "stocks_only");

assert.match(overviewPage, /selectedCashLabel/);
assert.match(overviewPage, /cashFieldForPlatform,/);
Expand Down
13 changes: 13 additions & 0 deletions tests/account_history_presentation_validation.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
chartUnavailable,
defaultOverviewChartAccountId,
filterAssetHistoryByRange,
presentAccountHoldings,
formatBinanceNativeQuantity,
formatBinanceWalletAmount,
formatLocalChangeTime,
Expand Down Expand Up @@ -393,3 +394,15 @@ assert.equal(
overviewLifecycleDetail(null, null, "今日周期记录未取得"),
"今日周期记录未取得,不能据此确认周期结果。",
);

// Read-only holdings: unknown/stale/unscoped/malformed is "empty" (暂无), never zero rows.
assert.deepEqual(presentAccountHoldings(null), { status: "empty" });
assert.deepEqual(presentAccountHoldings({ data_status: "fresh", positions: [] , positions_scope: "stocks_only" }), { status: "empty" });
assert.deepEqual(presentAccountHoldings({ data_status: "stale", positions: [{ symbol: "SOXL", quantity: "4", market_value: "1", currency: "USD" }], positions_scope: "stocks_only" }), { status: "empty" });
assert.deepEqual(presentAccountHoldings({ data_status: "fresh", positions: [{ symbol: "SOXL", quantity: "4", market_value: "1", currency: "USD" }] }), { status: "empty" });
assert.deepEqual(presentAccountHoldings({ data_status: "fresh", positions: [{ symbol: "SOXL", quantity: 4, market_value: "1", currency: "USD" }], positions_scope: "stocks_only" }), { status: "empty" });
assert.deepEqual(presentAccountHoldings({ data_status: "fresh", positions: [{ symbol: "TQQQ", quantity: "10", market_value: "900.5", currency: "USD" }], positions_scope: "strategy_symbols_only" }), {
status: "available",
scopeNote: "仅列出策略标的,不是账户全部持仓",
rows: [{ symbol: "TQQQ", quantity: "10", marketValue: "900.5", currency: "USD" }],
});
2 changes: 2 additions & 0 deletions web/strategy-switch-console/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,3 +430,5 @@ The existing promotion adapter alone accepts the valid subset of this exact part


Firstrade facts use `firstrade_account_snapshot_history.v1` / `firstrade_account_snapshot.v1` and a distinct optional `FIRSTRADE_ACCOUNT_FACTS_SYNC_TOKEN`. The binding workflow accepts `platform=firstrade` with one protected `FIRSTRADE_ACCOUNT_FACTS_BINDING_JSON` entry. It preserves other entries and rejects replacement of an existing conflicting binding. Each entry adds private `broker_account_id`; the public account selector may remain an alias. Native identity must match at ingress, storage, current read and history read; neither ACK nor authorized read models return it. Source kind is `deployment_runtime_account`. A snapshot contains one provider-reported currency/net-assets row and optional cash row with `cash_balance` / `source_tag=provider.cash_balance` in that same currency. Missing currency/equity is rejected; buying power, inferred USD and derived position totals are not accepted. Existing money, time, session and retention limits apply. The frontend includes fresh Firstrade assets and cash balance in configuration totals; absent cash stays missing. This interface does not enable collection or trading, configure private credentials, or prove live source adoption.

Holdings release order and rollback: [staged rollout runbook](../../docs/account-facts-holdings-rollout.md).
Loading
Loading