Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 26 additions & 7 deletions .github/scripts/open-collect-pr.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Open the PR for one collect.yml run from your own account:
# Open the PR for one collect.yml run as TechEngineBot:
#
# bash .github/scripts/open-collect-pr.sh <run-id>
#
Expand All @@ -8,15 +8,24 @@
# pr.json, in a collect-* artifact. Run this by hand from a TechAPI clone to
# push the run's changes to the task's collect/* branch and open the PR, or
# update the task's open one. It never touches your checkout or your branches.
# The PR comes from your account, so pr-checks runs on it like on any PR.
# Your account pushes the branch and TechEngineBot opens the PR, so pr-checks
# runs on it like on any PR.
#
# TechEngineBot's gh login lives in its own config directory, so your own gh
# login is never used for the PR or changed. Log it in once, signing in to
# GitHub as TechEngineBot in the browser:
#
# GH_CONFIG_DIR="$HOME/.config/gh-techenginebot" gh auth login -h github.com -p ssh --skip-ssh-key -s project -w
#
# Set TECHENGINEBOT_GH_CONFIG_DIR to keep it somewhere else.
#
# The artifact is the run's output, so nothing in it is trusted. GitHub's own
# record of the run, its title, says which task ran, and so which branch and PR
# it may update. Only the patch's changes are used: they are applied to the
# run's develop base in a scratch index, and may only add or change plain files
# under data/. The commit is made here, with the task's fixed title as its
# whole message, and the body loses every @ and issue reference before your
# account posts it.
# whole message, and the body loses every @ and issue reference before
# TechEngineBot posts it.
set -euo pipefail

run=${1:?usage: bash .github/scripts/open-collect-pr.sh <run-id>}
Expand All @@ -38,6 +47,16 @@ for c in python3 python; do
done
[ -n "$py" ] || { echo "Python 3 is needed on PATH, as python3 or python." >&2; exit 1; }

# Check TechEngineBot's login before anything is pushed.
bot_config=${TECHENGINEBOT_GH_CONFIG_DIR:-$HOME/.config/gh-techenginebot}
# A GH_TOKEN or GITHUB_TOKEN in the environment would win over that login.
bot() { env -u GH_TOKEN -u GITHUB_TOKEN GH_CONFIG_DIR="$bot_config" gh "$@"; }
if [ "$(bot api user --jq .login 2>/dev/null)" != TechEngineBot ]; then
echo "gh in $bot_config is not logged in as TechEngineBot. Log it in once with:" >&2
echo " GH_CONFIG_DIR=\"$bot_config\" gh auth login -h github.com -p ssh --skip-ssh-key -s project -w" >&2
exit 1
fi

# collect.yml titles each run "collect ingest <category>", "collect refresh" or
# "collect promote" from its inputs, and the run can't change that.
info=$(gh run view "$run" --repo "$repo" --json workflowName,event,headBranch,conclusion,displayTitle \
Expand Down Expand Up @@ -149,8 +168,8 @@ number=$(gh api "repos/$repo/pulls?state=open&base=develop&head=${repo%/*}:$bran
if [ -n "$number" ]; then
# gh pr edit can't set draft. A drafts run writes incomplete records, so its
# PR goes back to draft first, even if it was marked ready.
if [ "$draft" = true ]; then gh pr ready "$number" --repo "$repo" --undo; fi
gh pr edit "$number" --repo "$repo" --title "$title" --body-file "$dir/body.md" \
if [ "$draft" = true ]; then bot pr ready "$number" --repo "$repo" --undo; fi
bot pr edit "$number" --repo "$repo" --title "$title" --body-file "$dir/body.md" \
--add-assignee Seungpyo1007 --add-assignee TechEngineBot \
--add-label data --add-label enhancement \
--milestone "Massive dataset rebuild (1989-2026)"
Expand All @@ -161,5 +180,5 @@ else
--label data --label enhancement \
--milestone "Massive dataset rebuild (1989-2026)"
if [ "$draft" = true ]; then set -- "$@" --draft; fi
gh pr create "$@"
bot pr create "$@"
fi
5 changes: 3 additions & 2 deletions .github/workflows/collect.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,10 @@ run-name: "collect ${{ inputs.task }}${{ inputs.task == 'ingest' && format(' {0}
#
# The token is read-only. A run commits its data changes locally, runs the
# validators and the integrity gate, and uploads the commit as collect.patch in
# a collect-* artifact. A maintainer then opens the PR from their own account:
# a collect-* artifact. A maintainer then runs this by hand:
# bash .github/scripts/open-collect-pr.sh <run-id>
# so pr-checks runs on it like on any PR. Each task keeps at most one open PR,
# Their account pushes the task's branch and TechEngineBot opens the PR, so
# pr-checks runs on it like on any PR. Each task keeps at most one open PR,
# on a fixed collect/* branch that the script force-pushes and whose body it
# edits in place. Add the `final-check` label for the full build and the
# Tier 0-3 verification.
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,13 +73,17 @@ New and refreshed records come from
from the Actions tab on `develop`. It runs one of TechEngine's collectors
(`ingest`, `refresh` or `promote`) with a read-only token: the run validates
its changes and uploads them as a patch, and never pushes, opens a PR or posts
a comment. The maintainer then opens the PR from their own account, so
pr-checks runs on it like on any PR:
a comment. The maintainer then runs the helper below by hand. Their account
pushes the run's branch and TechEngineBot opens the PR, so pr-checks runs on
it like on any PR:

```bash
bash .github/scripts/open-collect-pr.sh <run-id> # from any TechAPI clone
```

TechEngineBot's gh login is kept apart from the maintainer's own. The helper
prints the one-time login command if it is missing.

Each task keeps one open PR on a fixed `collect/*` branch, updated in place by
later runs. Collection never runs on a schedule.

Expand Down