[jwrosewell]

Problem

On rc the file system watcher starts only when automatic updates are on. A deployment with automatic updates off that replaces its data file on disk gets no watcher, so the engine goes on with the data it loaded and the new file is not used. The published package watched the file in that case.

An on-premise engine that uses the default update URL turns autoUpdate off when it has no License Key (deviceDetectionOnPremise.js lines 502 to 512, ipIntelligenceOnPremise.js lines 357 to 367). So on rc every deployment with no License Key lost its watcher, although watching a file downloads nothing.

• main at 384bff2, dataFileUpdateService.js line 481, is if (dataFile.fileSystemWatcher) {. 384bff2 is the 4.5.102 tag, and line 481 of dataFileUpdateService.js in the published fiftyone.pipeline.engines 4.5.102 on npm is the same.
• rc at 871e7a3, dataFileUpdateService.js line 863, is if (dataFile.autoUpdate && dataFile.fileSystemWatcher) {. The condition came from #52, merged into rc on 10 November as aded0086, which also gave the watcher tests autoUpdate so that they would still get a watcher.

The release draft #83 cannot ship until this is back the way main has it.

The rule

The file system watcher runs on its own setting and needs no License Key. The License Key is needed only for the two things that download, which are polling and the update on start-up. This rule was set on 12 November, and the specification now states it, as quoted below.

What the specification says

specifications#143 changed the data update page to state the rule, and merged into the specification's rc on 12 November as 77dc1f62. data-updates.md there is identical to the draft's head d48add76, which this body quoted before the merge. The Automatic update master switch section, lines 99 to 109 says

The autoUpdate option is the master switch for Automatic update via HTTP (polling). When it is disabled, polling does not take place, however it is configured.

Automatic update from File (file system watching) does not depend on autoUpdate. The file system watcher runs on its own setting, whether autoUpdate is enabled or disabled, because watching a file downloads nothing.

A License Key is needed only for the two mechanisms that download, which are polling and the update on start-up. The file system watcher needs no License Key.

and the Automatic update from File section, lines 182 to 186 says

The file system watcher runs on its own setting, whether the autoUpdate master switch is enabled or disabled. Watching a file downloads nothing, so the watcher needs no License Key. An application with no License Key, such as one running the free Lite data file, can put a new file in place and the Aspect Engine refreshes from it.

Before #143 merged, the page on the specification's rc at 48e28393 said the opposite. The master switch section, lines 99 to 105 said "Automatic update from File (file system watching) does not take place, even if a file system watcher is otherwise configured", and the Automatic update from File section, lines 178 to 181 said "With autoUpdate disabled, no file system watcher is created and changes to the data file on disk are not detected automatically."

51degrees-go#110 made the same change in Go at e53aa8fc, with TestTheWatcherRunsOnItsOwnSetting holding the watcher to its own setting in three cases, and three tests losing the automatic update setting they had been given only to get a watcher. This draft follows it.

What changed

Two commits.

1. 26e6b6f6, the tests alone.
   • In dataUpdateSpecification.test.js the test "is not created while autoUpdate is off" held the old rule. It is replaced by "runs on its own setting" with the three cases of go#110, being watcher on with automatic updates on, watcher on with automatic updates off, and watcher off with automatic updates on. Each case checks for the "Creating file system watcher" log entry and then whether a changed file is loaded within three seconds. The third case waits the full three seconds to show that nothing loads the file.
   • "a file system watcher is logged" had autoUpdate, a polling interval and fake timers only to get a watcher, and loses them.
   • In dataFileUpdateService.test.js the watchedOptions and register helpers existed only to give the watched data files autoUpdate, and to keep the polling timer that autoUpdate starts from running. They are removed. The three watcher tests and the watched data file of the collection test go back to { fileSystemWatcher: true } and registerDataFile, as they are on main.
2. 98420f12, the fix. registerDataFile creates the watcher on fileSystemWatcher alone, and its comment says the watcher runs on its own setting whatever autoUpdate is, because watching a file downloads nothing.

Nothing about polling, the update on start-up or the License Key changes. The engines still turn autoUpdate and updateOnStart off with no License Key, and pass fileSystemWatcher through unchanged, so they need no change.

Red and green

• Red. Full build run 43297550986 on 26e6b6f, the tests commit alone on rc 871e7a3. Ubuntu_ARM_Node_24 failed with 5 failed, 2 skipped and 1339 passed, and the other five build jobs were cancelled when it failed. The five failures are

  1. "file system watcher › runs on its own setting, watcher on, automatic updates off", with Expected: true and Received: false at the check for the "Creating file system watcher" entry, so no watcher was created,
  2. "log messages › a file system watcher is logged", with Received array: [],
  3. "file watcher keeps watching after a failed refresh",
  4. "file watcher with no error listener writes a failed refresh to the console", and
  5. "file watcher with an error listener reports a failed refresh to it", the last three each exceeding the 10 second timeout waiting for a watcher event that never came.

  The other two cases of the new test passed.

• Green. The full build below, on the head 98420f1, passes on all six build jobs with 1344 tests passed, 2 skipped and none failed out of 1346. No test fails, so the five that failed in the red run pass, and so do all three cases of the new test.

Full build on the head

Run 43300455731, on the throwaway branch tmp/pr-matrix-90b for head 98420f12, success. Read from the job logs:

• All seven Setting 'PullRequestSha' lines, in Configure and the six build jobs, are 98420f1.
• Configure printed "Left out 2 macOS entries".
• Each of the six build jobs (Ubuntu, Ubuntu ARM and Windows, each on Node 22 and Node 24) has 1344 unit tests passed of 1346 with 2 skipped, 132 integration tests passed of 136 with 4 skipped, and lint with 0 errors and 188 warnings.
• Complete printed "Dry run - not merging".

The draft's two checks on the head, Only rc into main and utm-link-lint, are green. The tmp/pr-matrix-90 and tmp/pr-matrix-90b branches are deleted. The branch is 0 commits behind rc 871e7a3.

macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.

What remains

1. The body of the release draft #83 says in item 7 of "Before you upgrade" that "The file system watcher runs only while autoUpdate is on" and links #69. Once this draft is in rc that sentence is no longer true. The condition came from Update the data file as the specification says #52, not Write the data update messages and levels the .NET service writes #69.
2. The two automaticupdates/dataFileSystemWatcher.js examples still ask for a License Key and turn autoUpdate on, as they do on main. This draft leaves them alone.