feat: serve external plugins from nginx via multi-stage Docker build - #511
Conversation
|
@copilot |
Revalidation complete. Here is the gap analysis against all 10 acceptance criteria and the fix applied. All criteria were met except one critical bug:
Criterion 9 — bug fixed in commit The IED Editor entry in
Fixed by aligning with the consistent pattern used by all other plugins:
This path also works in local development mode: when the submodule is initialised, vite copies |
Done in commit
The Dockerfile |
| optional editor metadata: `--icon`, `--kind`, `--active-by-default`, | ||
| `--require-doc`. To compute the hash beforehand: | ||
|
|
||
| ```sh |
There was a problem hiding this comment.
is this relevant for us, specially here? Eventually I would add a section troubleshooting with this information. not everybody will have curl installed ...
There was a problem hiding this comment.
not relevant, i deleted the line
…r build Add a JSON configuration file, a download/validation shell script, and a multi-stage Dockerfile so that external plugins are fetched and integrity- checked at container build time and then served directly by the CoMPAS nginx service — no browser requests to external sources required. - remote-plugins.json: lists each plugin URL, local destination path and optional SHA-256 digest - scripts/download-plugins.sh: downloads every plugin with curl; fails the build if a download fails or a provided SHA-256 does not match - Dockerfile: new plugin-downloader Alpine stage before the nginx stage; layer ordering ensures the download cache is only invalidated when remote-plugins.json actually changes - docs/how-to/remote-plugins.md: explains the config format, how to add or update plugins, and security recommendations Closes #510
- Print the "no SHA256" warning to stderr so it is visible in build logs - Add REQUIRE_SHA256 env-var / Docker build-arg: when true, the build fails immediately for any plugin that has an empty sha256 field - Expose ARG REQUIRE_SHA256 in the Dockerfile plugin-downloader stage - Document the new flag in docs/how-to/remote-plugins.md
The IED Editor entry in remote-plugins.json referenced IedEditor.js, a local re-export shim that only exists in the repository's packages/external-plugins/ directory and is NOT present on the oscd-editor-ied deploy branch. This caused curl --fail to receive a 404 and fail the Docker build. Fix by aligning with the consistent pattern used by all other plugins: download the natural bundle file (oscd-editor-ied.js) directly from the deploy branch and serve it under its natural path, and update plugins.js to reference /external-plugins/oscd-editor-ied/oscd-editor-ied.js. This also matches local development: when the submodule is initialised, vite copies packages/external-plugins/oscd-editor-ied/** → dist/external- plugins/oscd-editor-ied/, so the same path works in both modes.
Per issue #510, all container-related files are now placed under the distribution/ directory: - Dockerfile → distribution/Dockerfile - remote-plugins.json → distribution/remote-plugins.json - scripts/download-plugins.sh → distribution/scripts/download-plugins.sh Update Dockerfile COPY paths to reference distribution/ prefix (build context remains the repo root). Add file: distribution/Dockerfile to both workflow files. Update docs to reference new paths.
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
…secure Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
…nt flags Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
Updated sha256 values to empty strings for all plugins so that when updates are deployed always the version is taken without build issues. Signed-off-by: David Monichi <david.monichi@bearingpoint.com>
Signed-off-by: David Monichi <david.monichi@bearingpoint.com>
Signed-off-by: David Monichi <david.monichi@bearingpoint.com>
3b6acee to
2b9dade
Compare
| { | ||
| name: 'IED', | ||
| src: '/external-plugins/IedEditor.js', | ||
| src: '/external-plugins/oscd-editor-ied/oscd-editor-ied.js', |
There was a problem hiding this comment.
I think this might break the IED plugin, because the oscd-editor-ied doesn't have a default export (which CoMPAS expects). So /external-plugins/IedEditor.js' was added as a workaround:
export { OscdEditorIED as default } from './oscd-editor-ied/oscd-editor-ied.js'
There was a problem hiding this comment.
Yes, the old path has been restored.
| { | ||
| name: 'Plugins Hub', | ||
| src: '/external-plugins/compas-bearingpoint-plugins/plugins/plugins-hub/index.js', | ||
| icon: 'details-icon', | ||
| activeByDefault: true, | ||
| kind: 'editor', | ||
| requireDoc: true, | ||
| }, | ||
| { | ||
| name: 'Engineering Wizard', | ||
| src: '/external-plugins/compas-transnetbw-plugins/plugins/engineering-wizard/index.js', | ||
| icon: 'star', | ||
| activeByDefault: true, | ||
| kind: 'editor', | ||
| requireDoc: true, | ||
| }, | ||
| { | ||
| name: 'Template Generator (TransnetBW)', | ||
| src: '/external-plugins/compas-transnetbw-plugins/plugins/template-generator/index.js', | ||
| icon: 'copy_all', | ||
| activeByDefault: true, | ||
| kind: 'editor', | ||
| requireDoc: true, | ||
| }, | ||
| { | ||
| name: 'Location Manager', | ||
| src: '/external-plugins/compas-transnetbw-plugins/plugins/location-manager/index.js', | ||
| icon: 'edit_location_alt', | ||
| activeByDefault: false, | ||
| kind: 'editor', | ||
| requireDoc: true, | ||
| }, | ||
| { | ||
| name: 'Location Viewer', | ||
| src: '/external-plugins/compas-transnetbw-plugins/plugins/location-viewer/index.js', | ||
| icon: 'location_on', | ||
| activeByDefault: false, | ||
| kind: 'editor', | ||
| requireDoc: true, | ||
| }, | ||
| { | ||
| name: 'History Viewer', | ||
| src: '/external-plugins/compas-transnetbw-plugins/plugins/history-viewer/index.js', | ||
| icon: 'history', | ||
| activeByDefault: false, | ||
| kind: 'editor', | ||
| requireDoc: true, | ||
| }, | ||
| { | ||
| name: 'Archive Explorer', | ||
| src: '/external-plugins/compas-transnetbw-plugins/plugins/archive-explorer/index.js', | ||
| icon: 'archive', | ||
| activeByDefault: false, | ||
| kind: 'editor', | ||
| requireDoc: true, |
There was a problem hiding this comment.
I guess these plugins only work within the container and not when running locally via pnpm start? Could we add a way to populate /external-plugins for local dev too?
There was a problem hiding this comment.
Sure, a few things have been added:
- a new script
npm run download:plugins, which executes/distribution/scripts/download-plugins.shlocally; - a new target in
vite.config.tsto copy the downloaded files into/dist/external-plugins.
Note: to execute the shell script in a Windows environment, Git Bash or WSL is needed.
| "name": "plugins-hub", | ||
| "url": "https://ase-compas.github.io/compas-bearingpoint-plugins/compas/plugins/plugins-hub/index.js", | ||
| "dest": "compas-bearingpoint-plugins/plugins/plugins-hub/index.js", | ||
| "sha256": "" |
There was a problem hiding this comment.
When building the new docker image locally and testing it, I noticed the Plugin store fails to load any registries:
GET /proxy/sprinteins.github.io/oscd-plugin-store/plugins.json 404 (Not Found)
GET /proxy/ase-compas.github.io/compas-transnetbw-plugins/plugins.json 404 (Not Found)
GET /proxy/ase-compas.github.io/compas-bearingpoint-plugins/plugins.json 404 (Not Found)
Not sure if that's just an issue in my environment or if there's something missing in the nginx config?
There was a problem hiding this comment.
In local development, Plugins Hub requires a configured wiremock proxy running in background (more info here: DEVELOPMENT.md). For this, the nginx config must be also extended with more locations (only for local environment), however I'm not exactly sure if I should do it in this PR.
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
…c copy Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
7e7b352 to
9ce2793
Compare
Signed-off-by: Stiliyan Valkanov <stiliyan.valkanov@bearingpoint.com>
|
github-project-automation
Bot
moved this from Todo
to Done
in PR Overview (excl dependabot)
External plugins were pulled from third-party URLs at browser runtime via git submodules. This replaces that approach with build-time download and integrity verification, so the CoMPAS nginx container serves all plugins itself with no outbound browser requests.
Changes
distribution/remote-plugins.json(new)Declarative config listing all 9 external plugins (previously git submodules) with
url,dest(path under/external-plugins/), and optionalsha256:{ "plugins": [ { "name": "Publisher", "url": "https://raw.githubusercontent.com/com-pas/oscd-publisher/deploy/oscd-publisher.js", "dest": "oscd-publisher/oscd-publisher.js", "sha256": "" } ] }distribution/scripts/download-plugins.sh(new)POSIX shell script run during Docker build:
curl --fail— build fails on download errorREQUIRE_SHA256=trueenv var to make the build fail for any plugin without a hash (strict mode for production)distribution/Dockerfile(multi-stage)plugin-downloaderAlpine stage:COPY distribution/remote-plugins.json→ run download script.remote-plugins.jsonis copied before theRUNstep so the download layer is only cache-invalidated when the plugin config changes, not on app code changes.COPY --from=plugin-downloader /build/external-plugins /usr/share/nginx/html/external-pluginsARG REQUIRE_SHA256for strict-mode buildsfile: distribution/Dockerfile(build context remains the repo root)docs/how-to/remote-plugins.md(new)Documents config schema, how to compute and add SHA-256 hashes, and the
REQUIRE_SHA256flag.Notes
Dockerfile,remote-plugins.json,scripts/download-plugins.sh) are placed under thedistribution/directory.sha256fields indistribution/remote-plugins.jsonare intentionally empty; they must be populated before production use (see docs forcurl | sha256sumworkflow)./usr/share/nginx/html; no nginx config changes needed.