Skip to content

Run every example on the free data files, and show what needs the Enterprise data file - #32

Merged
jwrosewell merged 5 commits into
rcfrom
feature/examples-free-data-files
Nov 9, 2026
Merged

jwrosewell merged 5 commits into
rcfrom
feature/examples-free-data-files

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Every example must run out of the box with the free data files, and show the properties that only the Enterprise data file has, with a link to ask for a copy where they are not available. The rule is going into the required examples pages of the specification in specifications#79. The same change was merged into Go's rc in 51degrees-go#55, and is a draft for .NET in 51degrees-dotnet#74, Java in 51degrees-java#48, Python in 51degrees-python#43 and PHP in 51degrees-php#35.

A run on a GitHub runner on 5 November showed where Node stood (run 42063810109, from rc at ed5b502). It built both native modules as the readmes say and ran each on-premise example from its own folder with no settings.

  • Device Detection. The examples downloaded the Lite file and ran, but showed only the properties Lite has, with no link. The web page showed HardwareVendor, HardwareName and DeviceType as "Unknown (the property 'hardwarevendor' is not in the results, so the current resource key does not include it)", a message written for the cloud service, and the client hints page as "Not found in datafile". The data file update example stopped with "Error: No license key available" and a stack trace.
  • IP Intelligence. Getting started and offline processing printed "Unknown (Property iprangestart not found in ip)" for every property only the Enterprise data file has, and the web pages did the same. Compare, metrics, suspicious and the mixed example looked for an Enterprise data file first and stopped with "RangeError: Maximum call stack size exceeded" while searching the disk for it. Translation listed every country alphabetically under "most probable country first". The data file update example stopped as the Device Detection one did.
  • Both. The data file age warning showed for data files published three and four days earlier. It, the free-file warning and the update examples linked to pricing.

Change

Device Detection, in device-detection/fiftyone.devicedetection.onpremise/examples/onpremise:

  • exampleUtils.js gains NEEDS_ENTERPRISE ("needs the Enterprise data file"), hasProperty, getEnterpriseValue, getEnterpriseNote, isLiteDataFile and getEnterpriseOffer. hasProperty reads the data file's own property list from the native engine, rather than the engine's properties, which are only those it was built to give.
  • Getting started shows HardwareVendor, HardwareName and DeviceType, offline processing writes them, and both web pages show them, each as needing the Enterprise data file when the data file does not have them, with the link. Offline processing writes the note as a # comment line at the top of its YAML output.
  • Metadata, match metrics and performance end with what the Enterprise data file adds and the link when they run on the Lite file.
  • Without a licence key the data file update example shows the Lite data file it would start from, says an update needs a licence key, gives the link and finishes. The automatic update examples link to the contact page for a licence key.

IP Intelligence, in ip-intelligence/fiftyone.ipintelligence.onpremise/examples/onpremise:

  • exampleUtils.js gains the same helpers, with isFreeDataFile, which tells the free files apart from the Enterprise data file by their having no IpRangeStart, and lacksEnterpriseProperties, which says which properties an example needs and where to ask for a copy when the data file has none of them.
  • Getting started, offline processing, the mixed example and the web pages show each property the data file lacks as needing the Enterprise data file, with the link. The web pages show their contact message, which they showed only for the tier 'Lite', a tier no IP Intelligence data file has, and say the country lists need the Enterprise data file in place of every country in alphabetical order.
  • Compare, metrics, suspicious and the mixed example use the Lite data files by default. Compare, metrics, suspicious and translation need properties only the Enterprise data file has, so with a free data file each says which, gives the link and stops.
  • Performance downloads the free ASN data file when it finds no data file, as decided on 5 November and recorded in specifications#79, and reads AsnName when the data file has no RegisteredName. Metadata and performance say what the Enterprise data file adds.
  • Without a licence key the data file update example shows the ASN data file, says an update needs a licence key, gives the link and finishes. Its check that stops an Enterprise data file being downloaded over a free one compared the tier with 'Lite', and now uses isFreeDataFile.

Both packages:

  • The data file age warning added 30 to the published date as text rather than adding 30 days. It now warns only for a data file more than 30 days old, and links to the contact page.
  • The free-file warning in the data file log is removed, as each example now says what the Enterprise data file adds in its own output.
  • The file search recursed through every folder up to the root of the disk. It now keeps a list of the folders still to search, so a large tree cannot overflow the stack, and skips a folder it cannot read.
  • A missing free data file is downloaded with PowerShell 7 when it is on the path, and otherwise with Windows PowerShell started without the module path it would inherit from PowerShell 7, as in the other languages. Inside GitHub Actions the examples no longer send examples-51degrees-node as the download source, so the data repository's script names the build and test runs are not counted as people using the examples.
  • run_examples.md and the package readmes say the examples need no settings and what they show on the free files. The IP Intelligence readme lists the on-premise examples.

Tests:

  • examplesWithoutSettings.test.js in each package's examples/onpremise folder runs every on-premise console example in its own process, from its own folder, with the data file and licence key settings removed, and checks that it finishes, shows what needs the Enterprise data file and gives the link. It also requests each web page in the test process on the free files.
  • CI puts the Enterprise IP Intelligence data file under the Lite file's name, and a fuller Device Detection data file under the Lite name in the tests folder. The Device Detection examples find the Lite file themselves in cxx/data/device-detection, where CI links the Lite asset. The IP Intelligence examples are given the free Lite file, which ci/fetch-assets.ps1 now fetches as the 51Degrees-LiteIpiV41.ipi asset and links into cxx/data/ip-intelligence. Without an IP Intelligence key the asset is also linked under the Lite name, so it is not downloaded twice.
  • The existing translation test returns early when the example stops on a free data file. In CI it is given the Enterprise data file and still checks the order.

Each contact link is https://51degrees.com/contact-us with utm_source=code, utm_medium=example, utm_campaign=51degrees-node, the example's file as utm_content, and utm_term=enterprise-data-file (or licence-key for the update examples).

Not changed

  • The web pages pass the current time as the data file's published time, so their own age warning never shows. That is left for its own change.
  • The Device Detection web page shows Device Id as "Unknown (the property 'deviceid' is not in the results, so the current resource key does not include it)", on rc as with this change. That is left for its own change.
  • The IP Intelligence web pages and the mixed API example have no Find out more section, which every demo should end with. That is left for its own change.
  • The mixed API example returns JSON, so it is unchanged apart from the file search no longer stopping it.
  • The match metrics example's documentation link is relative, /documentation/4.5/_device_detection__hash.html, and so cannot be followed from console output. It is left as it is.
  • The cloud examples.

Checks

  • The common-ci UTM link lint passes on the changed files.
  • ESLint, with the repository's configuration, passes on the examples and the new tests with no warnings.
  • A check on a GitHub runner from this change, before the two comment changes in the last commit (run 42072312857) built both native modules and ran every on-premise example from its own folder with no settings. Every example finished with exit code 0. The Device Detection examples showed HardwareVendor, HardwareName and DeviceType as needing the Enterprise data file, or said what the Enterprise data file adds, with the link. The IP Intelligence examples showed each missing property the same way, compare, metrics, suspicious and translation said which properties they need and stopped, and performance downloaded the ASN file and counted AsnName for all 20,000 addresses. Both update examples showed the free data file and said an update needs a licence key. The four web pages marked the missing properties, the IP Intelligence ones showed their contact message, and the mixed API example started, where on rc it stopped with the stack overflow.
  • In the same run the new tests passed 26 of 26, 11 for Device Detection and 15 for IP Intelligence. Run against the examples on rc, all 26 failed, so the tests detect the change.
  • The existing example tests passed in that run, apart from the match metrics test of the Enterprise data file, which looks for 51Degrees.hash, a file CI links in and that run did not have. It passed in the full build.
  • The full pull request build of 0611ff2 (run 42074443442) passed on all 8 legs on its second attempt. On the first attempt the Ubuntu ARM Node 24 leg failed in location/tests/geoLocationCloud.test.js, whose "No evidence error message" test timed out after 5 seconds waiting for the cloud service, and the other legs were cancelled. This change does not touch the location package, and that test passed on the second attempt.
  • On 8 November rc was merged into this branch (4ba380b), bringing in the changes rc gained since ed5b502, among them usage sharing (Share only the evidence usage sharing accepts unless asked for all #35). The full pull request build of 4ba380b (run 42493388288) passed on all 8 legs on its first attempt, being Ubuntu x64 and arm64, Windows and macOS with Node 22 and 24.
  • #47, which moves the IP Intelligence examples on the Enterprise file to LowMemory, is stacked on this branch and merges after it. Retarget Use LowMemory in the IP Intelligence examples built for the Enterprise file #47 to rc before this branch is deleted, or GitHub closes it.
  • In the Ubuntu Node 22 leg the integration tests ran 85 tests, with 81 passed and 4 skipped, where the full build of Allow only http and https in the callback URL, and test the zone and port rules #31 (run 41975470582), which rc now holds as ed5b502, ran 59 with 55 passed and 4 skipped. So the 26 new tests ran and passed, and they passed on the Windows, macOS and ARM legs too. The Selenium contract tests passed against the cloud web example and the on-premise one, which runs on the TAC data file.

Release note

The on-premise examples now run on the free data files with no settings, show the properties only the Enterprise data file has as needing it, with a link to ask for a copy, and no longer stop with an error when that file or a licence key is missing. The IP Intelligence performance example downloads the free ASN file rather than the Lite file when it finds no data file, and the compare, metrics, suspicious and translation examples say what they need and stop on a free data file.

On the free Lite data file, which a fresh clone downloads, the getting
started example showed only the properties Lite has, and the web page
showed HardwareVendor, HardwareName and DeviceType as "Unknown (the
property 'hardwarevendor' is not in the results, so the current resource
key does not include it)", a message written for the cloud service.
Getting started, offline processing and both web pages now show each
property only the Enterprise data file has as needing the Enterprise data
file, with a link to ask for a copy. Metadata, match metrics and
performance end with what the Enterprise data file adds and the same link
when they run on the Lite file.

Without a licence key the data file update example threw "No license key
available" with a stack trace. It now shows the Lite data file it would
start from, says that an update needs a licence key, gives the link and
finishes. The automatic update examples link to the contact page for a
licence key, in place of pricing.

The shared helpers gain a check of whether the data file has a property,
read from the data file's own list rather than the engine's, the text and
notes the examples write, and a check of whether the data file is Lite.
The data file age warning added 30 to the published date as text rather
than adding 30 days, so it showed for a data file published four days
earlier, and it now links to the contact page. The free-file warning in
the data file log is removed, as each example now says what the
Enterprise data file adds in its own output. The file search recursed
through every folder up to the root of the disk and overflowed the stack
when the file was not there. It now keeps a list of the folders still to
search and skips any it cannot read. Downloading a missing data file uses
PowerShell 7 when it is on the path, and otherwise Windows PowerShell
without the module path it would inherit from PowerShell 7. Inside GitHub
Actions the examples no longer send their download source, so the data
repository's script names the build and test runs are not counted as
people using the examples.
On the free Lite data file, which a fresh clone downloads, getting started
and offline processing printed "Unknown (Property iprangestart not found
in ip)" for every property only the Enterprise data file has, and the web
pages did the same. They now show each one as needing the Enterprise data
file and give a link to ask for a copy. The web pages also show their
contact message, which they showed only for the tier 'Lite', a tier no IP
Intelligence data file has. Metadata and performance say what the
Enterprise data file adds.

Compare, metrics and suspicious looked for the Enterprise data file, and
the mixed example for a fuller Device Detection data file, before the Lite
ones, and each overflowed the stack searching the disk for a file that
was not there. They now use the Lite data files by default. Compare,
metrics, suspicious and translation need properties only the Enterprise
data file has, so with a free data file each says which, gives the link
and stops, where suspicious printed false and 0 as if they were answers
and translation listed every country alphabetically under most probable
first. The translation test returns early when the example stops.

The performance example downloads the free ASN data file when it finds no
data file, rather than the Lite file, and reads AsnName when the data file
has no RegisteredName. Without a licence key the data file update example
threw "No license key available". It now shows the ASN data file, says
that an update needs a licence key, gives the link and finishes. Its check
that stops an Enterprise data file being downloaded over a free one
compared the tier with 'Lite', and now checks for the Enterprise
properties. The automatic update examples link to the contact page for a
licence key.

The shared helpers change as the Device Detection ones do, and gain a
check of whether the data file is free and a stop for an example that
needs properties only the Enterprise data file has.
CI puts the Enterprise IP Intelligence data file under the Lite file's
name, and a fuller Device Detection data file under the Lite name in the
tests folder, so nothing tested the free files a fresh clone uses. Each
package gains a test that runs every on-premise console example in its
own process, from its own folder, with the data file and licence key
settings removed, and checks that it finishes, shows what needs the
Enterprise data file and gives the link to ask for a copy. The web
examples are checked in the test process on the free files.

The Device Detection examples find the Lite data file themselves, in
cxx/data/device-detection, where CI links the Lite asset. The IP
Intelligence examples are given the free Lite data file, which CI now
fetches as the 51Degrees-LiteIpiV41.ipi asset and links into
cxx/data/ip-intelligence, and the performance and update examples find
the ASN data file themselves. Without an IP Intelligence key the Lite
asset is also linked under the Lite name, so it is not downloaded twice.
The tests run with the integration tests, which run every test under
examples.
The comment on NEEDS_ENTERPRISE said every property the examples ask for
is in the Enterprise data file, which nothing in the examples checks. The
web page test gave the wrong reason for removing usage sharing, which is
removed so that the test sends nothing to 51Degrees.
@jwrosewell
jwrosewell marked this pull request as ready for review November 9, 2026 00:13
@jwrosewell
jwrosewell merged commit 6f949e8 into rc Nov 9, 2026
2 checks passed
@jwrosewell
jwrosewell deleted the feature/examples-free-data-files branch November 9, 2026 00:13
@jwrosewell jwrosewell mentioned this pull request Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant