Repository navigation
Give each package its own version and release only what changed - #37
Draft
jwrosewell wants to merge 9 commits into
Draft
jwrosewell wants to merge 9 commits into
jwrosewell wants to merge 9 commits into
Conversation
Each of the 16 packages is listed in ci/packages.json and tagged <package>/<version>. common-ci's steps/release-plan.ps1 works out which packages a publish releases. A package is released when a file under its folders changed since its last tag, and every package depending on it is released with it. Only the patch number moves. ci/build-package.ps1 packs only the planned packages, each at its own version, and requires each package here at the version it has once the release is out. ci/publish-package.ps1 publishes only those tarballs and tags each package. ci/check-version-bump.ps1 and ci/check-release-interval.ps1 check each planned package on its own. Step 3.1 of 51Degrees/common-ci#306.
ci/release-plan.ps1 looks at COMMON_CI before the common-ci root, and ci/check-version-bump.ps1 runs the shared check twice for each package.
common-ci's release plan now leaves out documentation, tests, examples and LICENSE files, and calls a repository's ci/release-plan-filter.ps1 for the files left. This repository's filter does not count a package.json or remote_package.json whose only changes since the package's last tag are to description, keywords, homepage, repository, bugs, author, license, files or readme. A change to any other field, such as version, dependencies, main, exports or scripts, counts. Every package also ignores __tests__ folders, the other folder name jest finds tests in. The two shared packages export modules from their tests and examples folders, so they turn the defaults off and ignore only their Markdown and LICENSE files. ci/release-plan-filter.Tests.ps1 holds the Pester tests of the filter.
This was referenced Nov 9, 2026
fiftyone.devicedetection.shared and fiftyone.ipintelligence.shared published the modules in their examples and tests folders through index.js, so the release plan could not ignore those folders for them. The modules now live in lib, the packages use the default ignore list, and a test checks the layout.
# Conflicts: # ip-intelligence/fiftyone.ipintelligence.shared/index.js # ip-intelligence/fiftyone.ipintelligence.shared/lib/exampleOutput.js # ip-intelligence/fiftyone.ipintelligence.shared/types/lib/exampleOutput.d.ts
Each old path under examples and tests is a one-line re-export of the module now in lib, and is listed in files so that it is published. The release plan still ignores both folders, and the stubs never change, so the code in lib decides whether a package is released.
…ports fixed A change to files in package.json decides what a package ships, so it counts towards a release. The layout test now names the six re-exports, so deleting one fails, and each re-export carries the licence header. AGENTS.md no longer calls the shared packages an exception.
Draft
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Step 3.1 of common-ci#306.
ci/build-package.ps1line 35 runsnpm versionwith the run's one version for every package, so all 16 packages published 4.5.102 on 4 November 2026 (Publish run 41918901313) whatever had changed. That spends every package's allowance under the weekly release rule.The decisions in the run book apply. Each package has its own version, tagged
<package>/<version>, continuing from the version it last published, and only the patch number moves. A package is released when a file under its folders changed since its last tag, and every package that depends on it is released with it, with its requirement raised to the new version.Change
ci/packages.jsonlists the 16 packages withName,Path,ExtraPaths,DependsOnandPublish, read by common-ci'ssteps/release-plan.ps1from common-ci#307.DependsOnis the packages here that eachremote_package.jsonrequires. Both on-premise packages havecxxinExtraPaths, because their native modules compile from it.examplesfolder is in a field of its own,ExcludePaths, so that a change only to examples releases nothing, which is check 4 of the run book. The examples sit inside the package folders here, unlike in 51degrees-python, where they are a package of their own that is never published. The twosharedpackages keep theirexamplesfolders, because theirindex.jsexports the modules in them.location/docsis excluded too, as it is the only documentation folder inside a package.ci/release-plan.ps1loads common-ci's step from the common-ci root that the shared workflows run these scripts from. It gives git eachExcludePathsfolder as an exclude pathspec after the package's own, and it adds the build plan and the versions the requirements are set from. Only the publish scripts load it, so the pull request builds, which runci/build-project.ps1, do not need common-ci's step.ci/get-next-package-version.ps1gives the run the highest planned version as its label, and the newci/package-update-required.ps1prints the plan and answers 1 when it is empty, as common-ci'snightly-publish.configure.ps1expects.ci/build-package.ps1packs only the planned packages, each at its own version. Each requirement on a package here becomes^and that package's version once the release is out, which is its planned version, or its last released version when it is not in the plan. The build stops when a manifest andDependsOndisagree. When nothing is planned, every package is packed at its released version, as 51degrees-python does, so the nightly publish still tests packed packages.ci/install-package.ps1builds a native module from source when its package has no tarball, because that package is not being released.ci/publish-package.ps1publishes only the planned tarballs, then tags each package and makes one GitHub release per package with its tarball attached, in place of common-ci's single tag.ci/check-version-bump.ps1, modelled on 51degrees-dotnet's, runs common-ci'ssteps/check-version-bump.ps1for each planned package against its own tags, and again against the version npm'slatestdist-tag gives, so that a tag pushed by hand or aFirstVersioncannot set where a package starts.ci/check-release-interval.ps1runs common-ci'ssteps/check-release-interval.ps1with-TagPrefixfor each planned package. 51degrees-dotnet has no such script onrc, so this follows the comment above that step in common-ci'snightly-publish.yml.ci/packages.ps1loses$PublishedPackages, which nothing reads now, andAGENTS.mddescribes the scheme in place of the single version.Both check scripts run only once this repository is in the
$OwnChecklists of common-ci'snightly-publish.yml. Until then the shared check compares the run's label with the plain tags, which still refuses a minor move of any package, because the label is the highest planned version.Before this reaches
mainThe publish scripts need
steps/release-plan.ps1, which is on common-ci'srconly, and the workflows here take common-ci frommain. So common-ci's release of common-ci#307 tomainhas to come before the release of this repository'srcintomainthat carries this change. Merging intorcfirst changes nothing that runs, because the publish runs onmain.The first release
Step 4 of the run book. Before the first publish that runs these scripts, each package is tagged at the version it last published, so that each continues from there. All 16 published 4.5.102 from commit
384bff2d, which the Package job of run 41918901313 cloned before it published the 16 packages and pushed tag4.5.102, and npm'slatestis 4.5.102 for every one of them. The tags, all on384bff2d52cacba2e3315fdaaa1eca74a58be6ec, are:They should be lightweight tags with no GitHub release, for example
gh api -X POST repos/51Degrees/51degrees-node/git/refs -f ref=refs/tags/fiftyone.pipeline.core/4.5.102 -f sha=384bff2d52cacba2e3315fdaaa1eca74a58be6ec. The weekly check dates an annotated tag by when it was made and a release by when it was published, which would hold every package for a week from the day of tagging. A lightweight tag with no release is dated by its commit, 4 November 2026. The scripts now onmainread only plain tags such as4.5.102, so the tags can be pushed at any time before that publish. No tag has been pushed.Checks
Run 42368202986 laid the repositories out as common-ci's publish does, with common-ci's
rc(853b913d, the merge of common-ci#307) at the root and this branch inside it, made the 16 tags above locally and ran the publish's scripts in order. The plan it printed:Since 4.5.102,
rchas changedfiftyone.pipeline.core(#31) andfiftyone.pipeline.engines.fiftyone(#35), and 13 packages depend on one of them.fiftyone.pipeline.did,fiftyone.devicedetection.sharedandfiftyone.ipintelligence.sharedstay at 4.5.102.WEEKLY_RELEASES_FROMis not set.^4.5.103, andfiftyone.devicedetection.sharedandfiftyone.ipintelligence.sharedat^4.5.102, their last release.npm publish --dry-runfor the 13 tarballs, tagged each<package>/4.5.103locally, pushed nothing and set common-ci'sSkipUpdateTag.The same run then made local commits on top of tags at one commit, every package at 4.5.102 with
fiftyone.geolocationat 4.5.110 andfiftyone.devicedetection.cloudat 4.5.107, and checked each plan. Every check passed.fiftyone.geolocationplanned it alone at 4.5.111, and one tofiftyone.ipintelligence.translationplanned it alone at 4.5.103.fiftyone.pipeline.enginesplanned it and its 9 dependents, each after the packages it depends on. A change tofiftyone.pipeline.coreplanned 13 packages. A move of thecxxsubmodule planned both on-premise packages,fiftyone.devicedetectionandfiftyone.ipintelligence. A change to the exported examples offiftyone.devicedetection.sharedplanned it and its 3 dependents.fiftyone.devicedetection.cloud4.5.108 andfiftyone.geolocation4.5.111 among them. The tarballs of thefiftyone.pipeline.enginesplan requiredfiftyone.pipeline.core, which was not planned, at^4.5.102, andfiftyone.devicedetection.cloudat^4.5.108. With a tagfiftyone.pipeline.engines/4.6.0pushed by hand, the plan gave 4.6.1, which passed against the package's tags and was refused against npm's 4.5.102 as a minor move.ci/options.json, a workflow,README.md,AGENTS.md, threedocsfolders, a productreadme.mdand the examples of five packages planned nothing, andci/package-update-required.ps1answered 1.Run 42368758379 checked the publish's Test job for a plan of one package. With a change to
fiftyone.geolocationalone, the build packedfiftyone.geolocation-4.5.103.tgzonly, andci/install-package.ps1installed it from that tarball, leftfiftyone.pipeline.corelinked to the source, and built both native modules from source.Both runs tested the first commit,
27823384. The second commit,b863ad0e, corrects two comments and changes no code, and run 42369537155 repeats the dry run on it.No package was built for real and nothing was published or pushed. The temporary branches are deleted once their runs finish.
The release rule of 7 November
Commit b6d4f8d applies the decision that a package is not released when the only changes under its folders since its last tag are to documentation, tests, examples or manifest metadata such as links. It needs common-ci#310, which leaves out
**/*.md,**/docs/**,**/tests/**,**/test/**,**/examples/**and**/LICENSE*unless a package setsIgnoreDefaultsto false, adds each package'sIgnoreglobs, and callsci/release-plan-filter.ps1with the files left.ci/release-plan-filter.ps1does not count apackage.jsonorremote_package.jsonwhose only changes since the tag are todescription,keywords,homepage,repository,bugs,author,license,filesorreadme. A change to any other field counts,version, the dependencies,main,exportsandscriptsamong them, as do a manifest added or removed and one that is not JSON. Fields are compared as values with their keys sorted, so a change only to the layout or the order of the fields does not count.Ignoreholds**/__tests__/**, the other folder name jest finds tests in. No package has one today.sharedpackages hold their shipped modules inlib, not inexamplesandtests, so they use the default ignore list like every other package. Theirindex.jsused to exporttests/keyUtils.js,tests/testConstants.js,tests/exampleOutput.jsand the modules inexamples, which is why they had setIgnoreDefaultsto false.__tests__/layout.test.jsin each package checks the layout. No file in the repository imports the old paths,index.jsis themainentry and the only documented way in, and the generatedtypesmoved with the code. A deep import of an old path such asfiftyone.devicedetection.shared/examples/dataExtensionstops working, and no re-export was left because it would have to sit insideexamples.ci/release-plan-filter.Tests.ps1holds 25 Pester tests of the filter. Nothing in this repository runs them yet, so run them withInvoke-Pester ./ci/release-plan-filter.Tests.ps1.Run 42396135517 checked out common-ci at
feature/release-plan-ignore(7eb3d89, the head of common-ci#310) and this branch with the 16 tags pushed on 7 November. The Pester tests passed, 25 of 25. The plan at HEAD was the 13 packages above, with the reasonschanged since 4.5.102 (2 files)forfiftyone.pipeline.coreand(6 files)forfiftyone.pipeline.engines.fiftyone, andci/package-update-required.ps1answered 0. Each of 14 local commits, reset after each, planned what it should.fiftyone.pipeline.didpackage.jsonhomepageonlyfiftyone.pipeline.didpackage.jsondescriptionandkeywordsfiftyone.pipeline.didpackage.jsondependencyfiftyone.pipeline.didpackage.jsonversionfiftyone.pipeline.didpackage.jsonscriptsfiftyone.pipeline.didpackage.jsonhomepageandusage.jsfiftyone.pipeline.dida test, the readme and an examplefiftyone.pipeline.dida new__tests__folderfiftyone.pipeline.didremote_package.jsonhomepageonlyfiftyone.pipeline.didremote_package.jsondependencyfiftyone.devicedetection.sharedlib/keyUtils.jsfiftyone.devicedetection.sharedreadme.mdonlyfiftyone.ipintelligence.sharedlib/keyUtils.jsfiftyone.ipintelligence.sharedlib/dataExtension.jsShared packages hold their code outside examples and tests (9 November)
James Rosewell decided that examples, tests and docs must never cause a package release.
fiftyone.devicedetection.sharedandfiftyone.ipintelligence.sharedshipped modules from their ownexamplesandtestsfolders throughindex.js, so the default ignore either skipped a real change to shipped code or, with the defaults off as before, released the package for an example change.dataExtension.js,exampleConstants.jsandoptionsExtension.jsfromexamples, andexampleOutput.js,keyUtils.jsandtestConstants.jsfromtests, into alibfolder in each package, with the generatedtypes. It removesIgnoreDefaultsand the extra**/*.mdand**/LICENSE*entries from both packages inci/packages.json, so both use the default ignore list like every other package.tests/exampleOutput.jsfor IP Intelligence arrived onrcafterwards, and the merge ofrc(98a1a02) puts it inlibtoo.examples/keyUtils.jsholdingmodule.exports = require('../lib/keyUtils');, sorequire('fiftyone.devicedetection.shared/examples/keyUtils')and the other five paths in each package return the same module as before.filesinpackage.jsonlistslib, exactly those twelve files (six per package),*.jsandtypes. The release plan still ignoresexamplesandtests, which is right because the files never change, and the code inlibdecides whether a package is released. The generatedtypesare not duplicated at the old paths, so only the JavaScript paths are kept.__tests__/layout.test.jsin each package checks thatindex.jsexports every module, thatindex.jsrequires nothing from anexamplesortestsfolder, thatnpm pack --dry-runlists everylibfile and the old-path files and nothing else underexamplesortests, that each old path returns the same module as the file inlib, and that each old file is a one-line re-export.examplesandtestsfolders did not exist.tscleaving thetypesunchanged,requirefrom the on-premise and IP Intelligence cloud packages returning every export, andeslintpassing.Setting 'PullRequestSha'71f7427 on both lines. All six Ubuntu, Ubuntu ARM and Windows jobs passed. macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.main.Outstanding
$OwnChecklists in common-ci'snightly-publish.yml, and common-ci#307 has to reach common-ci'smain, as above.readme.mdno longer releases it, because common-ci's defaults leave out**/*.md, so a README fix reaches npm with the package's next release for another reason.ExcludePathsand common-ci's default**/examples/**and**/docs/**now leave out the same folders, soExcludePathsand the code inci/release-plan.ps1that reads it could go.