You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add interactive graph visualization with SVG, PDF and Markdown export - #134
Add Graph.visualize() to inspect declared tables and relationships directly in notebooks.
Standard and Compact layouts include light and dark themes, schema details, relationship highlighting and optional sample values.
Export SVG, vector PDF or Markdown on the machine running Kumo Engine, with selectable save commands in the notebook toolbar and a static SVG fallback for saved HTML output.
Notebook display automatically uses environment-provided anywidget/ipywidgets in Snowflake Workspace and the existing HTML path elsewhere, without changing package requirements.
Schema definitions alone make it difficult to inspect relationships across multiple tables before training.
Hover-driven camera movement shifts rows beneath the pointer, while fixed light colors make schema details unsuitable for dark notebook displays.
Notebook host controls can also overlap graph actions placed at the top right.
Snowflake Workspace blocks the scripted HTML path, saved Jupyter outputs can strip scripts and inline SVG, and VS Code relocates executable scripts outside their original graph element.
The Fix
Render declared schemas in deterministic Standard/Compact layouts with stationary hover details, coordinated light/dark styling, manual camera controls, optional samples and sample-free SVG/PDF/Markdown exports.
Public API
Options and examples:
graph.visualize(
path=None, # str, pathlib.Path, or Noneshow_cols=False,
show_samples=False,
layout="auto", # "auto", "standard", or "compact"
)
Option
Behavior
path=None
Return SVG text with interactive notebook display support, without writing a file.
path="graph.svg"
Write UTF-8 SVG of the initial layout, including the displayed columns.
path="graph.pdf"
Write the vector diagram and complete paginated schema/connection index.
path="graph.md" or path="graph.markdown"
Write a Mermaid relationship diagram and all-column Markdown schema tables.
show_cols=False
Display PK, FK and declared time (TS) and end-time (TE) columns.
show_cols=True
Include other columns, up to 10 rows per table, or 5 above 10 tables; key/time/end-time columns always remain visible even above that limit.
show_samples=False
Read no source data.
show_samples=True
Add up to six non-null samples per visible column to notebook tooltips from a preview of up to 30 rows per table.
layout="auto"
Choose Standard or Compact from graph structure, independently of viewport size.
layout="standard"
Select the layered relationship layout.
layout="compact"
Select the compact arrangement.
Returned SVG: .as_widget()
Optional explicit widget conversion; Graph.visualize() still returns SVG text and selects the notebook transport at display time.
frompathlibimportPathfromIPython.displayimportdisplay# Notebook display; no source-data reads by default.graph.visualize()
graph.visualize(show_cols=True)
graph.visualize(show_samples=True)
graph.visualize(show_cols=True, show_samples=True, layout="compact")
# The same API automatically uses widget MIME in Snowflake Workspace.display(graph.visualize(show_cols=True))
# Explicit layout selection.graph.visualize(layout="auto")
graph.visualize(layout="standard")
graph.visualize(layout="compact")
# Save on the machine running Engine, including a training machine.graph.visualize("graph.svg", show_cols=True, layout="standard")
graph.visualize(path=Path("graph.pdf"), show_cols=True, layout="compact")
graph.visualize("graph.md")
graph.visualize("graph.markdown")
# The return value remains SVG text, including when a file is written.view=graph.visualize()
svg_text=str(view)
File suffix matching is case-insensitive; any suffix other than .pdf, .md or .markdown receives SVG.
Paths are relative to the current working directory unless absolute, parent directories must already exist, and existing files are overwritten.
PDF and Markdown indexes include every declared column regardless of show_cols; Markdown uses the viewer's Mermaid layout rather than the notebook arrangement, and its schema tables remain readable without Mermaid support.
The Export commands button reveals _ = graph.visualize("graph.pdf", layout="standard") first (using the selected layout) and _ = graph.visualize("graph.md") below it; executing a command writes to the notebook filesystem without a browser download or eager PDF generation.
The default HTML view follows browser appearance, the widget view inherits a declared host color scheme when mounted, and SVG/PDF files and the saved static image use the light palette with sample values excluded.
Notebook hosts and saved outputs
Context
Display and persistence contract
Databricks, local Jupyter and other HTML hosts
graph.visualize() or display(graph.visualize()); existing self-contained HTML path with Databricks nodarkmode, no anywidget import or comms
Snowflake Workspace
The same graph.visualize() and display(graph.visualize()) automatically publish actual widget MIME when the Workspace marker and active Snowflake notebook extension are present; one cached model supports independent frontend views, with no scripted HTML competing for display
Workspace packages
Runtime 2.6+ preinstalls anywidget/ipywidgets, including the documented GPU 2.9 target; older runtimes need compatible packages under their environment policy; missing or incompatible widget packages produce a static graph and actionable diagnostic
Connector extras
kumo-engine[snowflake] adds connector dependencies; package installation, connector choice and credentials do not select the notebook renderer
Saved HTML output
Complete, sample-free SVG image in the Python-requested initial layout plus “Rerun this cell to activate animation.”; hidden interactive controls until initialization succeeds; rerun once after upgrading and save to replace old output; no saved browser camera or layout-toggle state
Widget restoration and trust
Snowflake widget state is not persistent across sessions; HTML fallback does not repair a missing widget model; VS Code Restricted Mode can refuse HTML/SVG entirely
File retrieval and fullscreen
PDF/Markdown commands save in the notebook working directory for notebook-side retrieval; fullscreen remains host-permission-dependent with a clear denial message
Mac and Windows compatibility
Target
Scope and evidence
Mac/Windows browser connected to a hosted notebook
Native Engine dependencies run on the kernel host; no client-side Engine port needed, but actual macOS/Windows browser sessions remain untested
Native Apple Silicon Mac
macOS support already merged in #47; matching macOS wheel and native notebook smoke still required for this candidate; local validation artifact is Linux-only
Linux Chromium/Firefox/WebKit tests and cross-platform resolver checks; no native Mac/Windows installation, Safari-on-macOS or Windows browser certification; WSL is a separate Linux target
Blast Radius
Dependency declarations and uv.lock are unchanged from the PR base; graphs remain limited to 30 tables with valid declared schemas and no self-relations, PDF glyphs unavailable in the bundled fonts use explicit Unicode escapes, and notebook interaction depends on host HTML/widget/fullscreen permissions.
Sample previews can run warehouse queries and incur costs, their row limit does not bound bytes scanned, and saved notebook HTML or widget state can contain sampled values even though file exports and the static fallback image do not.
Verification
The graph/export tests and layout tests pass together (99 tests), including temporal-role retention beyond column limits, actual PDF/Markdown exports, PDF badge-spacing checks with bundled fonts, browser tooltip/visibility checks and scoped lint/import/whitespace checks.
Prior verification covers the complete installed wheel, actual IPython/kernel Workspace dispatch, Jupyter save/reopen, repeated-output VS Code renderer replay and genuine Databricks strict/theme/fullscreen checks; this end-time correction is checked against source without a new wheel or hosted deployment; host replays do not certify authenticated frontends or native Mac/Windows systems, and the existing restricted-BigQuery credential failures remain reproduced on the exact base commit's CI job.
Requested feature: inspect declared table relationships in notebooks, optionally preview samples, and export SVG, PDF or Mermaid/schema Markdown without graph ingestion or new base dependencies, with automatic Workspace widget transport and readable saved HTML output.
Reuse graph metadata, existing preview interfaces and Matplotlib already required by Kumo Core; no dependency or lockfile changes remain, and anywidget/ipywidgets are supplied by the notebook environment and imported only for widget display.
File
Function / scope
What changed
Why
example_notebooks/colab_example_notebook.ipynb
Graph setup example
Display the graph after schema validation.
Introduce visualization in the existing Colab workflow.
Display the graph through the same graph.visualize() API after schema validation.
Automatically select the Workspace widget channel using its observed active runtime fingerprint.
packages/kumo-engine/README.md
Visualization, persistence and shared widget recipe
Document temporal-column roles, runtime-local export commands, automatic Workspace routing and optional widget dependency contract and saved static output; share the same feature specification across transports.
Keep usage aligned with actual display, filesystem, sample and lifecycle behavior.
render temporal roles, SVG.ipython_display, SVG.as_widget, SVG._viewer and exports
Preserve time/end-time columns as TS/TE beyond display limits and in export indexes; reuse one viewer specification, select widget MIME at notebook display time in Workspace, cache the widget model, preserve the SVG string/static exports and remove eager PDF payloads.
Keep the plain API portable, prevent scripted HTML from competing with widget MIME, and avoid optional imports/comms until display.
Render vector cards/routes and complete schema pages in an opaque light palette, label TS/TE roles, reserve measured badge width for column text, expand tabs and skip wrapping lines that fit.
Provide readable PDF exports without adding libraries or omitting hidden columns.
Retain stationary hover and navigation; reveal PDF then Markdown commands; render a complete static SVG image and rerun note until live initialization succeeds.
Survive Jupyter saved-output sanitization and avoid disabled/truncated controls, browser-download failures and unnecessary PDF generation.
Require the observed Workspace marker and active notebook extension for automatic widget selection; reuse themes/disposal across HTML and lazy anywidget; isolate view IDs/state and resolve relocated scripts within their output container.
Preserve Databricks colors, avoid cross-output leaks/collisions, and initialize duplicated saved outputs correctly under VS Code script relocation.
packages/kumo-engine/kumo/engine/graph.py
Graph.visualize
Retain signature and suffix-based SVG/PDF/Markdown writes; document temporal-column retention, automatic Workspace display, optional conversion, saved samples and notebook-file export commands.
Keep return/API compatibility while accurately describing transport and filesystem behavior.
Existing graph/export parameters and automatic/optional widget regression
Verify temporal roles, cap/overlap/missing-end cases and PDF font bounds, static-image bytes and sample exclusion, hidden inactive controls, lazy missing/present/incompatible widget dependencies, actual IPython display and final-expression routing, negative host detection, model reuse and no eager PDF display work.
Cover the reproduced persistence and transport boundaries while retaining schema/export/encoding regressions.
Cover deterministic layouts, dense/sparse ranks, component packing and route clearance.
Verify geometry independently of browser camera transforms.
Verification: 99 source graph/layout tests; actual temporal PDF/Markdown exports and browser role/visibility checks; prior installed-wheel automatic IPython/ZMQ-kernel dispatch and dependency-failure checks; actual Jupyter save/session-shutdown/untrusted-reopen; 24 static plus 24 interactive browser cases; nine installed widget cases; separate-output and duplicate-HTML VS Code renderer replay; strict and theme/fullscreen Databricks renderer checks; scoped lint/import/whitespace checks.
The earlier deployed Databricks r2 notebook validated 18 saved outputs and two representative fullscreen cases; widget/static behavior was validated locally in the preceding candidate, and this temporal-role correction is source-tested without a new wheel or hosted deployment. Limits: Automatic Workspace selection is checked against the captain-provided active runtime fingerprint using real IPython display paths; a fresh live Workspace frontend run remains pending.
The detector uses observed runtime markers, not a public Snowflake host-detection API; package presence and connector extras alone never select it.
Missing widget models, VS Code Restricted Mode, host fullscreen restrictions and actual native Mac/Windows installation/browser tests are outside the demonstrated fix; platform import/resolver evidence identifies the native prerequisites. Stopping condition: publish this validated visualization candidate and the two existing PR artifacts only; no native Engine port, new PR/comment, host infrastructure change or CI monitoring.
The restricted BigQuery permission tests also fail on the exact base because their dedicated CI identities are absent; no connector/test bypass is included.
It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.
Graph.visualize renders declared table relationships as an SVG, with options for columns, sample previews, and layout. It supports notebook display and SVG, PDF, or Markdown file output. The change adds deterministic standard and compact layouts, an interactive notebook viewer, and shared widget lifecycle helpers. Example notebooks now display the graph after validation.
Sequence Diagram(s)
sequenceDiagram
participant Graph
participant Renderer as _visualization.render
participant Display as IPython.display.display
participant SVG
participant Viewer as _visualization_viewer.viewer
participant Widget as _widget.render_widget
Graph->>Renderer: Render graph tables and edges
Graph->>Display: Display returned SVG
Display->>SVG: Request notebook HTML representation
SVG->>Viewer: Build viewer markup
Viewer->>Widget: Build widget shell
Loading
Merge Risk: Low · previous review 53e89
Some exports may differ from the displayed graph or fail for unusual names, and widgets may not render on non-loopback HTTP notebooks. These bounded issues merit fixes or explicit acceptance before merging.
Pre-merge checks: 2 · 1
Failed checks: 1 inconclusive
Check name
Status
Explanation
Resolution
Change Map Design Review
The required <!-- kumo-pr-change-map --> conversation comment is not present in the supplied review context or repository checkout. The available PR description is truncated and only states that a c…
Provide the complete PR conversation comment beginning with <!-- kumo-pr-change-map -->, including the change table, linked problem, rationale, focused verification, and stopping condition. Re-run the design review with the full comment.
Passed checks: 2 passed
Check name
Status
Explanation
Pr Size
The pull request changes 13 files, based on the authoritative review-scoped diff. This does not exceed the 40-file limit.
Public Resource Access
The pull request adds one actionable external URL: packages/kumo-engine/README.md:130 links to Snowflake widget installation guidance. The base revision has no such link, and an anonymous request re…
Full details: Public Resource Access
Explanation
The pull request adds one actionable external URL: packages/kumo-engine/README.md:130 links to Snowflake widget installation guidance. The base revision has no such link, and an anonymous request returned HTTP 200 with the expected documentation. Other URL-like additions are SVG/XML namespace identifiers or excluded example fixtures. No package source URL or dependency repository location was added.
Full details: Change Map Design Review
Explanation
The required <!-- kumo-pr-change-map --> conversation comment is not present in the supplied review context or repository checkout. The available PR description is truncated and only states that a change map exists; it does not provide the table or its required failure, existing behavior, smallest change, verification, and stopping condition. The repository confirms that this comment is required for this review. Although the reviewed diff is available, its design cannot be assessed against the missing proposal or linked problem.
Resolution
Provide the complete PR conversation comment beginning with <!-- kumo-pr-change-map -->, including the change table, linked problem, rationale, focused verification, and stopping condition. Re-run the design review with the full comment.
Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
The reason will be displayed to describe this comment to others. Learn more.
Follow-up Comments
Keep graph labels on their allocated line.
144-144: Minor · Functional Correctness · Quick win
Keep graph labels on their allocated line.
If a table or column name contains \n, this value reaches axes.text as multiline text. Matplotlib draws an additional line, but the graph allocates space for one SVG text element. The PDF graph can therefore overlap nearby content. Replace embedded newlines with a single-line representation for graph text. Keep the original name in the schema index. (matplotlib.org)
287-287: Minor · Functional Correctness · Quick win
Clamp tooltips to the visible host bounds.
place() limits the tooltip width but does not clamp its horizontal position. In a narrow inline view, a centered card can select the left side and place the tooltip outside the .kumo-graph element. Because the host uses overflow: clip, schema and sample details can be hidden. Clamp x after route clearance, or choose the side with more available space before clamping.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Caution
Some comments are outside the diff and can’t be posted inline due to GitHub limitations.
⚠️ Outside diff range comments (1)
Minor · Treat literal newlines as hard breaks in _wrap. · _visualization_pdf.py:142-153
packages/kumo-engine/kumo/engine/_visualization_pdf.py:142-153 Minor · Functional Correctness · Quick win
Treat literal newlines as hard breaks in _wrap.
Graph.visualize() accepts names such as sales\ndata and order\nid. _wrap() keeps each value as one entry, so card sizing reserves one line. The PDF graph renderer passes that value to Matplotlib, which renders multiple lines. The extra line can overlap the next column row or footer. clip_on=False also lets it extend beyond the card boundary.
Split newline-separated text in _wrap() so sizing and SVG/PDF rendering use the same line count. Keep the existing schema-index handling unchanged.
Suggested fix
def _wrap(text: str, width: int) -> list[str]:
lines, line, size = [], '', 0
+ text = text.replace('\r\n', '\n').replace('\r', '\n')
split, split_size = 0, 0
for char in text:
code = ord(char)
if not (code in (9, 10, 13) or 32 <= code <= 0xD7FF
or 0xE000 <= code <= 0xFFFD or 0x10000 <= code <= 0x10FFFF):
raise ValueError('Table and column names must contain valid XML '
'characters to visualize them.')
+ if char == '\n':+ lines.append(line)+ line, size = '', 0+ split, split_size = 0, 0+ continue
char_size = 0 if combining(char) else (
2 if code > 127 or char.isupper() or char in 'mw@%&#' else 1)
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
501-501: Minor · Functional Correctness · Quick win
Preserve visible columns in the PDF command.
If the notebook displays graph.visualize(show_cols=True), this command reruns visualize() with its default show_cols=False. The saved PDF diagram therefore omits columns visible in the viewer, although its schema index remains complete. Carry the displayed show_cols setting into the generated command. (raw.githubusercontent.com)
packages/kumo-engine/kumo/engine/_widget.py: 1
129-129: Minor · Stability & Availability · Quick win
Generate view IDs without a secure-context-only API.
If a notebook runs on a non-loopback HTTP origin, crypto.randomUUID() is unavailable. Rendering then fails before host.innerHTML inserts even the static fallback. Generate the per-view ID with an API available in that context, such as crypto.getRandomValues(). (developer.mozilla.org)
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
Review complete—looks good. The notebook persistence issue is fixed in the latest tested revision (1d63aed): trusted and untrusted save/restart/reopen checks passed, along with 98 focused tests (1 optional-widget test skipped).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GitHub Actions
Inactive
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Summary
Add
Graph.visualize()to inspect declared tables and relationships directly in notebooks.Standard and Compact layouts include light and dark themes, schema details, relationship highlighting and optional sample values.
Export SVG, vector PDF or Markdown on the machine running Kumo Engine, with selectable save commands in the notebook toolbar and a static SVG fallback for saved HTML output.
Notebook display automatically uses environment-provided anywidget/ipywidgets in Snowflake Workspace and the existing HTML path elsewhere, without changing package requirements.
See the change map for file-by-file rationale.
What Happens and Why
Schema definitions alone make it difficult to inspect relationships across multiple tables before training.
Hover-driven camera movement shifts rows beneath the pointer, while fixed light colors make schema details unsuitable for dark notebook displays.
Notebook host controls can also overlap graph actions placed at the top right.
Snowflake Workspace blocks the scripted HTML path, saved Jupyter outputs can strip scripts and inline SVG, and VS Code relocates executable scripts outside their original graph element.
The Fix
Render declared schemas in deterministic Standard/Compact layouts with stationary hover details, coordinated light/dark styling, manual camera controls, optional samples and sample-free SVG/PDF/Markdown exports.
Public API
Options and examples:
path=Nonepath="graph.svg"path="graph.pdf"path="graph.md"orpath="graph.markdown"show_cols=FalseTS) and end-time (TE) columns.show_cols=Trueshow_samples=Falseshow_samples=Truelayout="auto"layout="standard"layout="compact".as_widget()Graph.visualize()still returns SVG text and selects the notebook transport at display time.File suffix matching is case-insensitive; any suffix other than
.pdf,.mdor.markdownreceives SVG.Paths are relative to the current working directory unless absolute, parent directories must already exist, and existing files are overwritten.
PDF and Markdown indexes include every declared column regardless of
show_cols; Markdown uses the viewer's Mermaid layout rather than the notebook arrangement, and its schema tables remain readable without Mermaid support.The Export commands button reveals
_ = graph.visualize("graph.pdf", layout="standard")first (using the selected layout) and_ = graph.visualize("graph.md")below it; executing a command writes to the notebook filesystem without a browser download or eager PDF generation.The default HTML view follows browser appearance, the widget view inherits a declared host color scheme when mounted, and SVG/PDF files and the saved static image use the light palette with sample values excluded.
Notebook hosts and saved outputs
graph.visualize()ordisplay(graph.visualize()); existing self-contained HTML path with Databricksnodarkmode, no anywidget import or commsgraph.visualize()anddisplay(graph.visualize())automatically publish actual widget MIME when the Workspace marker and active Snowflake notebook extension are present; one cached model supports independent frontend views, with no scripted HTML competing for displaykumo-engine[snowflake]adds connector dependencies; package installation, connector choice and credentials do not select the notebook rendererMac and Windows compatibility
Graphimport reaches the compiled DiskGraph extension before rendering; native portability #91 and experimental Windows wheel work #140 remain prerequisites, not support added by this visualization PRBlast Radius
Dependency declarations and
uv.lockare unchanged from the PR base; graphs remain limited to 30 tables with valid declared schemas and no self-relations, PDF glyphs unavailable in the bundled fonts use explicit Unicode escapes, and notebook interaction depends on host HTML/widget/fullscreen permissions.Sample previews can run warehouse queries and incur costs, their row limit does not bound bytes scanned, and saved notebook HTML or widget state can contain sampled values even though file exports and the static fallback image do not.
Verification
The graph/export tests and layout tests pass together (99 tests), including temporal-role retention beyond column limits, actual PDF/Markdown exports, PDF badge-spacing checks with bundled fonts, browser tooltip/visibility checks and scoped lint/import/whitespace checks.
Prior verification covers the complete installed wheel, actual IPython/kernel Workspace dispatch, Jupyter save/reopen, repeated-output VS Code renderer replay and genuine Databricks strict/theme/fullscreen checks; this end-time correction is checked against source without a new wheel or hosted deployment; host replays do not certify authenticated frontends or native Mac/Windows systems, and the existing restricted-BigQuery credential failures remain reproduced on the exact base commit's CI job.