Skip to content

Surface graph visuals in customer agent workflows - #154

Merged
devanj08 merged 4 commits into
mainfrom
docs/graph-visualization-skills
Nov 5, 2026
Merged

devanj08 merged 4 commits into
mainfrom
docs/graph-visualization-skills

Conversation

@devanj08

@devanj08 devanj08 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Give users a graph view before model training.
Generated notebooks include an interactive graph cell, and SDK scripts save an SVG alongside run outputs.
Agents can use Markdown schemas when comparing graph choices during model iteration.

What Happens and Why

The existing skills omit the native graph viewer and incorrectly list it as unavailable.

The Fix

One shared capability covers notebook visuals, script exports, and requested diagrams without adding a required review or pause.
The router, SDK references, and host packages expose it through existing workflows.
See the change map.

Blast Radius

Changes affect customer skill guidance and packaging.
Unavailable previews report their error and let modeling continue with the original graph.

Verification

All 184 skill tests, 14 renderer tests, three host package builds, and documentation, public-reference, and secret checks pass.
Executed notebook, SVG, and Markdown examples cover display dispatch, file output, and continuation after preview failures; hosted notebook rendering was not exercised.

@devanj08 devanj08 added the documentation Improvements or additions to documentation label Nov 2, 2026
@copy-pr-bot

copy-pr-bot Bot commented Nov 2, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@devanj08
devanj08 temporarily deployed to gha-kumo-workflow-tests November 2, 2026 17:00 — with GitHub Actions Inactive
@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

The change adds a graph visualization skill for notebook previews, SDK exports, and schema inspection. The data science skill routes graph-related requests to it and adds previews and SVG exports to training workflows. Graph API references now include .visualize(). Capability packaging includes the new skill, and a test checks that capability skills are packaged. Host runtime documentation describes graph display and export behavior across notebook environments.

Merge Risk: Low · up to 65a07

This change adds graph preview and export guidance for notebooks and SDK scripts. The remaining issue is narrow: a generated script could overwrite an existing graph.svg in its working directory. It is bounded and does not block merging, but the example should point at a unique file in the confirmed run output directory.

Pre-merge checks: Passed 2 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Change Map Design Review Inconclusive The required PR conversation comment starting with <!-- kumo-pr-change-map --> is not available. The supplied context summarizes a change map but does not provide its proposed design table or prose,… Provide the complete maintained change-map conversation comment, including its observed failure, requested release requirement, existing-behavior evidence, proposed changes, verification plan, and stopping condition. Then reassess the desig…
Passed Passed checks: 2 passed
Check name Status Explanation
Pr Size Passed The pull request changes 10 files. The count does not exceed the failure threshold of 40.
Public Resource Access Passed PASS. The only added external URL is https://github.com/NVIDIA/kumo-engine/pull/134 in agent-skills/skills/kumo-data-science/SKILL.md:16 and `agent-skills/skills/kumoai-reference/references/kumo_f…
Full details: Public Resource Access

Explanation

PASS. The only added external URL is https://github.com/NVIDIA/kumo-engine/pull/134 in agent-skills/skills/kumo-data-science/SKILL.md:16 and agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md:10; anonymous access returned HTTP 200. Added repository paths and package source locations resolve in the checkout, and the referenced source files are publicly readable from the NVIDIA GitHub repository. No added or modified reference is confirmed inaccessible.

Full details: Change Map Design Review

Explanation

The required PR conversation comment starting with &lt;!-- kumo-pr-change-map --&gt; is not available. The supplied context summarizes a change map but does not provide its proposed design table or prose, so necessity, abstraction-layer fit, behavior preservation, verification, and stopping-condition claims cannot be assessed against that comment.

Resolution

Provide the complete maintained change-map conversation comment, including its observed failure, requested release requirement, existing-behavior evidence, proposed changes, verification plan, and stopping condition. Then reassess the design against the repository diff.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 86eccb1 and 65a0711.

Files selected for processing: 5
  • agent-skills/skills/hosts/databricks-genie/runtime.md
  • agent-skills/skills/hosts/snowflake-coco/runtime.md
  • agent-skills/skills/kumo-data-science/SKILL.md
  • agent-skills/skills/kumo-graph-visualization/SKILL.md
  • agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@devanj08

devanj08 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor Author

Change map

The requested workflow gives users a graph view in notebooks and an SVG from scripts, with Markdown inspection available during model iteration.
One shared skill replaces the missing visualization guidance; examples, package builds, and tests validate it without running models or deploying skills.

File Function / scope What changed Why
agent-skills/README.md Capability inventory List graph visualization. Make the capability discoverable.
agent-skills/scripts/package_skill.py CAPABILITY Register the shared skill. Include it in all three host packages.
agent-skills/skills/hosts/databricks-genie/runtime.md Graph visualization Describe expected notebook display and file retrieval; mark hosted rendering unverified. Use the Databricks runtime.
agent-skills/skills/hosts/generic/runtime.md Graph visualization Describe notebook display and runtime-local files. Support local and remote Python environments.
agent-skills/skills/hosts/snowflake-coco/runtime.md Graph visualization Describe widget dependencies, static fallback, and file retrieval; mark hosted rendering unverified. Use the Snowflake notebook runtime.
agent-skills/skills/kumo-data-science/SKILL.md Routing and generation Add notebook visuals and script SVG exports; distinguish baseline bundle from viewer requirements. Provide visuals before training without a review gate.
agent-skills/skills/kumo-graph-visualization/SKILL.md Shared capability Cover user visuals and Markdown model iteration, with missing-API upgrade hints and typed failure diagnostics. Provide one concise workflow for every host.
agent-skills/skills/kumoai-reference/references/graph_construction.md Graph API Expose visualization and link the shared skill. Correct the API inventory and guide graph workflows.
agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md Graph reference Add the visualization signature, skill link, and source-build requirement. Keep generated calls aligned with the API.
agent-skills/tests/test_package_skill.py test_every_reference_resolves Check every capability in each built package. Catch missing packaged skills.

@devanj08 devanj08 changed the title Add graph inspection and visualization to customer skills Surface graph visuals in customer agent workflows Nov 2, 2026
@devanj08
devanj08 temporarily deployed to gha-kumo-workflow-tests November 2, 2026 19:44 — with GitHub Actions Inactive
@devanj08
devanj08 temporarily deployed to gha-kumo-workflow-tests November 2, 2026 19:58 — with GitHub Actions Inactive
@devanj08
devanj08 requested a review from kumo-isaac November 2, 2026 20:00
@devanj08
devanj08 marked this pull request as ready for review November 2, 2026 20:00

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

Other comments: 1
agent-skills/skills/kumo-graph-visualization/SKILL.md: 1

37-37: Minor · Data Integrity & Integration · Quick win

Write the graph export to a unique, confirmed run path.

Graph.visualize() overwrites an existing destination. Path('graph.svg').resolve() can therefore replace an existing user file when the script runs in a directory that already contains graph.svg. The fixed relative path also relies on the process current directory being the run's confirmed output directory, which this example does not establish. Use a unique filename under the confirmed output directory, or under the permitted scratch directory when the export is disposable.

Additional comments: 1
agent-skills/skills/hosts/snowflake-coco/runtime.md-204-204: 1

204-204: Minor · Functional Correctness · Quick win

Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Handle exports outside the Workspace directory.

An export under /tmp, which this runtime permits for artifacts, does not appear in the Workspace file browser. Tell users to copy it into the Workspace directory or upload it to a stage before retrieval. (raw.githubusercontent.com)


Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between fa46bed and 86eccb1.

Files selected for processing: 10
  • agent-skills/README.md
  • agent-skills/scripts/package_skill.py
  • agent-skills/skills/hosts/databricks-genie/runtime.md
  • agent-skills/skills/hosts/generic/runtime.md
  • agent-skills/skills/hosts/snowflake-coco/runtime.md
  • agent-skills/skills/kumo-data-science/SKILL.md
  • agent-skills/skills/kumo-graph-visualization/SKILL.md
  • agent-skills/skills/kumoai-reference/references/graph_construction.md
  • agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md
  • agent-skills/tests/test_package_skill.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@devanj08
devanj08 requested a review from 9Olive November 2, 2026 21:53
@9Olive

9Olive commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Looks good - skills match the merged implementation (#134) and the workflows ran cleanly locally. Three small changes before merge:

  1. Stale bundle pin. kumo-data-science/SKILL.md:15 and kumo_finetuning_sdk.md:14 still target 20261007+2e1eb12, which predates Graph.visualize(). Point them at a build that includes Add interactive graph visualization with SVG, PDF and Markdown export #134.

  2. Fallback. Catching AttributeError hides typos like graph.visualise() as "preview unavailable", and gives no upgrade hint on older packages. Suggest:

    if not hasattr(graph, 'visualize'):
        print('Graph preview needs kumo-engine with Graph.visualize (#134+); continuing.')
    else:
        try:
            display(graph.visualize())
        except Exception as error:
            print(f'Graph preview unavailable: {type(error).__name__}: {error}')

    Same pattern for the script export.

  3. Hosted rendering. databricks-genie/runtime.md:283 and snowflake-coco/runtime.md:203 state rendering behavior we haven't verified on either host. Add > **Not verified:** notes like the rest of those files. For Snowflake, note the widget needs anywidget/ipywidgets and falls back to a static SVG without them.

@kumo-isaac

Copy link
Copy Markdown
Contributor

Eval results: routing and codegen, base fa46bed vs head 86eccb1

Harness: kumo_skill_dev main @ eba83e0 with one local loader change: when the ref has kumo-graph-visualization/SKILL.md, add it to the codegen prompt. The router now tells every train notebook to take its cell from that file, and the loader's fixed file list would otherwise leave it out. Model: harness default (Cortex). Base and head were loaded through the same loader.

Intake / routing (90 cases, skill_v2_ga, with cards, Nemotron judge)

base head
Correct behavior 80/90 82/90
Rubric met 43/90 43/90
  • No case regressed. Two improved: nov-sales-regression-complete-with-output and sainsburys-complete-formal went from ask_back to the expected proceed behavior.
  • No training request was routed to graph visualization instead of intake.
  • An earlier judge pass showed head at 74%. That was HTTP 429 rate limiting from running two judges at once; the affected rows were re-judged.
  • Runs: run-20261103-172604 (base), run-20261103-172854 (head).

Codegen (38 cases, skill_v3_codegen, compare.py against specs.json)

base head
First-try spec match 38/38 36/38
query / calls / selection 38/38 38/38
Notebooks with the display(graph.visualize()) cell 0 38, all after graph.validate() and before the query/training
Notebooks that also write graph.svg 0 16

Re-ran the two failing cases 3× on each side:

Case base (4 runs) head (4 runs)
nov-sales-regression-known-context-with-output: pk_added (primary_key="STATUS_EVENT_ID" on event table rig_status_events) 0/4 3/4
videoamp-churn-known-context: dropped_column_kept (end_date) 0/4 1/4 (likely noise)

Findings

  1. Probable regression: a primary key gets declared on an event table. It reproduces in 3 of 4 head runs and none on base. A likely cause is the new skill's push toward declared "key and time roles" (the viewer needs declared keys), but I haven't confirmed that. Worth a line saying the visual must not change graph construction (no keys added for the viewer's sake).
  2. The script-only SVG export leaks into notebooks (16/38). The sentence "Include the SVG export … before training" follows "The user may ask for a script instead of a notebook", and the model applies it to notebooks too. Suggest wording such as "In a script (not a notebook), include the SVG export…". Harmless at runtime (try/except), but it writes files nobody asked for, e.g. /tmp/kumo-run/graph.svg.
  3. Graph.visualize exists on kumo-engine main (Add interactive graph visualization with SVG, PDF and Markdown export #134) with the signature documented here. Wheels built before Add interactive graph visualization with SVG, PDF and Markdown export #134 don't have it, including the 20261010+d9a703d eval bundle; the cell falls back to "Graph preview unavailable". No GPU run was done.

🤖 Generated with Claude Code

@kumo-isaac

Copy link
Copy Markdown
Contributor

Follow-up: fixes for findings 1 and 2, tested; finding 3 confirmed

Proposed change (4 lines, kumo-graph-visualization/SKILL.md only)

 Reuse the graph for the current task; graph changes follow `kumoai-reference/references/graph_construction.md`.
+The visual shows the graph exactly as built and never changes it: add no primary key, column, or edge for the viewer.
+Tables without a primary key, such as event tables, display as they are.
 ...
 Continue the workflow normally; pause for visual review only if the user asks.
+A notebook displays the graph only; it saves no SVG unless the user asks for a file.

 ## Generated SDK scripts

+This section applies to scripts only, never to notebooks.

Why the first line is accurate: render() only requires a primary key on edge destination tables, which Graph already enforces, so keyless event tables render fine.

Results

base fa46bed head 86eccb1 head + change
Codegen spec match (38) 38/38 36/38 38/38
nov-sales-regression-known-context-with-output, no PK on event table (4 runs) 4/4 1/4 4/4
videoamp-churn-known-context, end_date dropped (4 runs) 4/4 3/4 4/4
Notebooks with the preview cell 0 38 38
Notebooks saving graph.svg 0 16 0
Intake prompt — 94f094938e37 94f094938e37 (identical, so intake unchanged)

Do not reword the router line instead. I first also changed the router sentence to "A script, never a notebook, includes the SVG export…". Codegen was equally clean, but intake regressed: nov-missing-positive-interaction and videoamp-missing-positive-interaction asked back 0/3 times, against 7/8 and 8/8 on base and head combined. Keeping the router unchanged avoids that.

Not covered: the eval has no script-output cases, so the script SVG path is untested by the eval.

Finding 3, confirmed: no released build has Graph.visualize

Build visualize
kumo-engine 1.0.0, the only PyPI version (tag kumo-engine-v1.0.0, 2026-10-17) no; neither the method nor kumo/engine/_visualization.py exists
main after #134 (2026-10-30), version 1.0.0.dev0 yes, unreleased
Fat bundles 20260917+d6d753e (staged) and 20261010+d9a703d (GPU eval) no

Until a release includes #134, every notebook and script generated from this PR prints "Graph preview unavailable". It's harmless, but the feature won't be visible to anyone yet.

🤖 Generated with Claude Code

@devanj08
devanj08 temporarily deployed to gha-kumo-workflow-tests November 3, 2026 18:51 — with GitHub Actions Inactive
@devanj08

devanj08 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 65a0711: both examples check for the API and report typed failures, and hosted rendering is marked unverified with Snowflake widget dependencies and static fallback documented. No released build includes #134 yet, so the version guidance distinguishes the historical baseline bundle from the source build required for visualization.

@devanj08
devanj08 merged commit 8bb1f51 into main Nov 5, 2026
109 of 112 checks passed
@devanj08
devanj08 deleted the docs/graph-visualization-skills branch November 5, 2026 21:49

This branch was previously deployed

1 inactive deployment
gha-kumo-workflow-tests — 65a0711d Deployed Nov 3, 2026 by devanj08 via import #1365
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants