Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 50 additions & 6 deletions .github/workflows/fee-schedule-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,15 @@ name: Fee schedule check
# notify: when rates changed, the check failed, or the PR could not be opened. Opens or updates one
# issue. Rates are never edited automatically.
#
# Only GITHUB_TOKEN is used. Pull requests opened with it do not start other workflows, so the
# bump job runs the relevant tests itself and a reviewer closes and reopens the PR to run CI.
# The bump job pushes its branch and opens the PR with a short-lived installation token of the
# GitHub App qsl-qpk-pin-sync (variable FIRSTRADE_PR_APP_ID, secret FIRSTRADE_PR_APP_PRIVATE_KEY),
# so CI starts on the PR like on any other. The token is scoped to this repository with only
# contents and pull-requests write. The repository setting that lets Actions create pull requests
# stays off.
# Fallback: when the App variable or secret is missing, or the token cannot be created, the job
# keeps using GITHUB_TOKEN. It pushes the branch, cannot open the PR, and the notify job opens or
# updates an issue with a compare link. A PR opened with GITHUB_TOKEN would not start other
# workflows anyway, so the bump job runs the relevant tests itself.
# Optional repository variable FEE_CHECK_CONTACT: a monitored address that is sent to sec.gov in
# the User-Agent (SEC fair-access policy).

Expand Down Expand Up @@ -104,15 +111,38 @@ jobs:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
REPORT: ${{ needs.check.outputs.report }}
# Non-secret flag: secrets cannot be tested directly in `if:`.
APP_CONFIGURED: ${{ vars.FIRSTRADE_PR_APP_ID != '' && secrets.FIRSTRADE_PR_APP_PRIVATE_KEY != '' }}
# The bump rewrites a module that it imported; a stale .pyc would hide the change from pytest.
PYTHONDONTWRITEBYTECODE: '1'
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false

- name: Create GitHub App installation token
id: app_token
if: env.APP_CONFIGURED == 'true'
continue-on-error: true
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ vars.FIRSTRADE_PR_APP_ID }}
private-key: ${{ secrets.FIRSTRADE_PR_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: FirstradePlatform
permission-contents: write
permission-pull-requests: write

- name: Note the fallback to GITHUB_TOKEN
if: steps.app_token.outputs.token == ''
run: |
echo "::warning::No GitHub App token (variable or secret missing, or token creation failed). Falling back to GITHUB_TOKEN: the branch is pushed and the issue carries a compare link."

- name: Bump verified_through and open a pull request
id: bump
env:
# Empty when the fallback applies. Only the push and the PR creation use it.
APP_TOKEN: ${{ steps.app_token.outputs.token }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
Expand Down Expand Up @@ -150,7 +180,22 @@ jobs:
git commit -q -m "chore(fees): extend fee schedule verified_through to $new_date" \
-m "Scheduled fee-schedule-check found the SEC Section 31 and FINRA TAF rates unchanged." \
-m "No rate row was changed. Refs #497."
git push origin "$branch"
# Prefer the App token so that CI starts on the PR; never print it.
if [ -n "$APP_TOKEN" ]; then
GH_TOKEN="$APP_TOKEN" git push origin "$branch" || {
echo "::warning::Push with the App token failed; retrying with GITHUB_TOKEN."
APP_TOKEN=""
git push origin "$branch"
}
else
git push origin "$branch"
fi

if [ -n "$APP_TOKEN" ]; then
ci_note="CI starts on this pull request normally because the GitHub App opened it."
else
ci_note="CI did not start on this pull request because it was opened with \`GITHUB_TOKEN\`: close and reopen it to run the full checks."
fi

{
echo "## AI assistance"
Expand All @@ -170,13 +215,12 @@ jobs:
echo
echo "Workflow run: $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID."
echo "\`pytest tests/test_statement_reconciliation.py tests/test_check_fee_schedule.py\`"
echo "passed in that run. CI did not start on this pull request because it was opened"
echo "with \`GITHUB_TOKEN\`: close and reopen it to run the full checks."
echo "passed in that run. $ci_note"
echo
printf '%s\n' "$REPORT" | tail -n +2 | sed 's/^## /### /'
} > "$RUNNER_TEMP/pr_body.md"

if pr_url="$(gh pr create --base main --head "$branch" \
if pr_url="$(GH_TOKEN="${APP_TOKEN:-$GH_TOKEN}" gh pr create --base main --head "$branch" \
--title "chore(fees): extend fee schedule verified_through to $new_date" \
--body-file "$RUNNER_TEMP/pr_body.md")"; then
echo "result=pr-opened" >> "$GITHUB_OUTPUT"
Expand Down
22 changes: 17 additions & 5 deletions docs/statement-reconciliation.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,23 @@ failure is an error, never a pass.
titled "Firstrade fee schedule check needs attention". Rates are never edited
automatically; update the segments in a normal pull request.

The workflow uses `GITHUB_TOKEN` only. Pull requests it opens do not start CI, so
close and reopen the pull request to run the checks before merging. If the
repository setting that lets Actions create pull requests is off, the job leaves
the branch on the remote and the issue carries a compare link instead. Set the
repository variable `FEE_CHECK_CONTACT` to a monitored address: sec.gov rejects
The `bump` job pushes its branch and opens the pull request with a short-lived
installation token of the GitHub App `qsl-qpk-pin-sync` (ID 5004823), so CI starts on the
pull request like on any other. The App is already installed on this repository with
`contents: write` and `pull-requests: write`. The token is created with
`actions/create-github-app-token`, scoped to this repository only, and limited to
`contents: write` and `pull-requests: write`. The repository setting that lets Actions
create pull requests stays off. The setup is repository level:

- the repository variable `FIRSTRADE_PR_APP_ID` (the App ID);
- the repository secret `FIRSTRADE_PR_APP_PRIVATE_KEY` (the App private key).

Without the variable or secret, or when the token cannot be created, the job falls back to
`GITHUB_TOKEN`: it leaves the branch on the remote and the issue carries a compare link
instead; a pull request opened by hand from it needs CI to be run by closing and
reopening it.

Set the repository variable `FEE_CHECK_CONTACT` to a monitored address: sec.gov rejects
clients that do not identify themselves. The script is tied to the current page
layouts and to the filings named in `REVIEWED_FINRA_FILINGS`; when a layout
changes the check fails loudly and the parser needs an update.
Expand Down
2 changes: 1 addition & 1 deletion docs/statement-reconciliation.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

`verified_through`(当前 `2026-12-01`)是费率最后一次对照公开来源核实的交易日。之后的交易日为 `unsupported`(`..._not_verified_after_<日期>`):SEC 2027 财年 Section 31 费率尚未公布(FY2026 费率在 2027 财年拨款法案生效后 60 天内继续有效),上述 FINRA 备案虽即时生效但仍可能被 SEC 暂停。更新来源核实时需在同一变更中推进 `verified_through`。

`Fee schedule check` 工作流(`.github/workflows/fee-schedule-check.yml`)每月 22 日 UTC 02:17 自动执行这项复核,也可手动触发(`workflow_dispatch`):`scripts/check_fee_schedule.py check` 读取 SEC Section 31 公告和 FINRA 费率表、备案列表与备案 PDF,与费率表比对;抓取或解析失败一律视为错误,不会当作通过。全部一致时 `bump` 任务把 `verified_through` 推到下月末(并同步测试里的复核日期和文档),跑完测试后开 PR,不会自动合并。费率有变化、出现新的 FINRA 费用备案、SEC 页面提到暂停或检查失败时,`notify` 任务开或更新一个 issue,费率不会被自动修改。仅使用 `GITHUB_TOKEN`,它开的 PR 不会触发 CI,合并前请关闭再重新打开 PR 以运行检查。建议把仓库变量 `FEE_CHECK_CONTACT` 设为可联系的邮箱,sec.gov 会拒绝不表明身份的请求。
`Fee schedule check` 工作流(`.github/workflows/fee-schedule-check.yml`)每月 22 日 UTC 02:17 自动执行这项复核,也可手动触发(`workflow_dispatch`):`scripts/check_fee_schedule.py check` 读取 SEC Section 31 公告和 FINRA 费率表、备案列表与备案 PDF,与费率表比对;抓取或解析失败一律视为错误,不会当作通过。全部一致时 `bump` 任务把 `verified_through` 推到下月末(并同步测试里的复核日期和文档),跑完测试后开 PR,不会自动合并。费率有变化、出现新的 FINRA 费用备案、SEC 页面提到暂停或检查失败时,`notify` 任务开或更新一个 issue,费率不会被自动修改。`bump` 任务用 GitHub App `qsl-qpk-pin-sync`(ID 5004823)的短期安装令牌推送分支并开 PR(`actions/create-github-app-token`,仅限本仓库,只有 `contents: write` 和 `pull-requests: write`),所以 PR 会像其他 PR 一样触发 CI;该 App 已安装在本仓库,具有 `contents: write` 和 `pull-requests: write`;仓库里“允许 Actions 创建 PR”的设置保持关闭。配置在仓库级:仓库变量 `FIRSTRADE_PR_APP_ID`(App ID)和仓库 secret `FIRSTRADE_PR_APP_PRIVATE_KEY`(App 私钥)。变量或 secret 缺失、或令牌创建失败时,退回使用 `GITHUB_TOKEN`:只推送分支,issue 里带比较链接,手动开出的 PR 需关闭再重新打开才会运行检查。建议把仓库变量 `FEE_CHECK_CONTACT` 设为可联系的邮箱,sec.gov 会拒绝不表明身份的请求。

## 估值口径差异

Expand Down
Loading