Skip to content

feat(workflow): match search and listings against the copy on show - #8576

Draft
yangzhang75 wants to merge 3 commits into
apache:mainfrom
yangzhang75:pin/4-search
Draft

yangzhang75 wants to merge 3 commits into
apache:mainfrom
yangzhang75:pin/4-search

Conversation

@yangzhang75

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

A listing is where most people meet a workflow, so it has to agree with what opening it shows.
This makes search and the hub match, display and link against the copy on show.

  • Search matches the copy the caller may see. One search returns both the caller's own
    workflows and public ones, so the filter is a disjunction of guarded filters — the working copy
    for rows they were granted, the frozen copy for rows they reach only because those are public.
    Written over bare columns rather than a CASE, so each side stays eligible for its own PGroonga
    index. A pinned workflow no longer turns up on keywords that exist only behind the pin, and the
    author still finds their own work on what they just typed.
  • Listings show the frozen name and description to a viewer without granted access — the same
    copy the detail page serves, so a card cannot advertise a title that opening it does not show.
    Both are NULL while following, which leaves the live values in place.
  • The hub is the public shelf, so everything on it is listed as the public sees it, for the
    author too: they are looking at the shelf, not at their own dashboard.
  • One definition of "the public is behind". WorkflowPublishService.pinDiffersFromWorkingCopy
    is the share dialog's comparison as a SQL condition, and the search projection and the hub
    listing both use it, so a card and the dialog cannot disagree about whether edits are held back.

Any related issues, documentation, discussions?

Closes #7940
Part of #7828. Design discussion: #7128. Stacked on #7853 and #8575; until those merge this PR
shows their commits too, and the review here is the last one.

How was this PR tested?

22 new cases in WorkflowPublishSpec plus WorkflowSearchQueryBuilderSpec and
UnifiedResourceSchemaSpec (559 in the dashboard package):

  • search not matching a public workflow on anything that exists only in unpublished edits, and
    matching it on its published copy — content, name and description, each separately;
  • an unpinned public workflow still matching the author's latest, and the author still matching
    their own workflow on a name only they can see;
  • a listing showing a public viewer the published name and description, a collaborator the live
    ones, and a private workflow's own listing untouched;
  • the drift flag a card carries: true while a pin holds edits back, false for a workflow with
    nothing frozen, and equal to what the share dialog reports for the same workflow — including
    after a rename and after a change of view, the two fields most easily left out of one of the
    three places that answer this.

The shared condition was checked by mutation: dropping the default-view term from it turns the
card-versus-dialog case red.

scalafmtCheckAll clean.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5)

🤖 Generated with Claude Code

yangzhang75 and others added 3 commits September 16, 2026 12:36
A public workflow follows the author's latest content, as publishing has
always done. This adds the other state: the author pins the version they
have now, and the public copy stops moving until they pin again.

`is_public` stays the on/off switch; `published_content` is the pin, NULL
while following. `WorkflowPublishService` owns the two states, and three
endpoints expose them: POST and DELETE `/workflow/pin/{wid}` to pin and
unpin, GET `/workflow/publish-status/{wid}` for what the author is shown.
Publishing and unpublishing move through the same service, so unpublishing
drops the pin rather than leaving a private workflow carrying one.

Two paths are narrowed so a pin can hold. A save wrote the whole row back,
so a publish landing while a save was in flight was silently rolled back,
and a request body could set the publish columns itself; saves now write
only name, description and content. Creating a workflow clears the publish
columns for the same reason.

Nothing reads the pinned copy yet: every workflow is in the following
state it is in today, and nothing on screen changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With a version pinned, a workflow has two copies: the author's working
copy and the frozen one on public show. This routes every read that
serves a viewer without granted access through the frozen copy, and
freezes the name and description with the graph.

`WorkflowPublishService.publicCopyOf` returns the three fields as a
group, so a surface cannot pick up the published graph under a title the
author has not published; `WorkflowAccessResource.hasGrantedAccess` is
the seam that decides which copy a caller gets. Granted access -- owner,
shared, project member -- keeps tracking the author's latest, because
sharing is not publishing.

Name and description freeze because they are as public as the graph: if
only the graph froze, a report about a title could be answered by editing
the title while the pinned copy still advertised it.

Routed through it: opening a workflow, the hub's read, Clone, Duplicate,
`/workflow_name`, `/workflow_description` and the size a listing shows.
A workflow that follows the author's latest -- every workflow today --
is served exactly what it is served now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Search and the listings it feeds were reading the author's live columns,
which for a pinned workflow is the one copy the public cannot open. A
draft would turn up in a public search under a title nobody has seen, and
the card would advertise a name the detail page does not show.

Each filter is now applied to whichever copy the caller may see:
`onVisibleCopy` builds the same filter twice -- over the live columns for
rows the caller was granted access to, over the frozen ones for rows they
reach only because the workflow is public -- and ORs the two. A
disjunction over bare columns rather than a CASE, so each side stays
eligible for its own fulltext index. Unpinned public rows fall back to
the live columns, so a following workflow searches exactly as it does now.

Listings carry two more things from the same query: the frozen name and
description to show a viewer without granted access, and whether the copy
on show is behind the author's working copy. `constructWhereClause` takes
`includePublic` for this; the other builders accept and ignore it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @aglinxinyuan, @tanishqgandhi1908
    You can notify them by mentioning @aglinxinyuan, @tanishqgandhi1908 in a comment.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 1 better · 🔴 6 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main 471e53c benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 452 0.276 20,129/35,033/35,033 us 🔴 +31.1% / 🔴 +133.9%
🔴 bs=100 sw=10 sl=64 1,104 0.674 84,259/120,045/120,045 us 🔴 +33.8% / 🔴 +20.1%
bs=1000 sw=10 sl=64 1,379 0.842 722,658/783,066/783,066 us ⚪ within ±5% / 🟢 +22.3%
Baseline details

Latest main 471e53c from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 452 tuples/sec 468 tuples/sec 839.58 tuples/sec -3.4% -46.2%
bs=10 sw=10 sl=64 MB/s 0.276 MB/s 0.286 MB/s 0.512 MB/s -3.5% -46.1%
bs=10 sw=10 sl=64 p50 20,129 us 22,677 us 11,937 us -11.2% +68.6%
bs=10 sw=10 sl=64 p95 35,033 us 26,721 us 14,976 us +31.1% +133.9%
bs=10 sw=10 sl=64 p99 35,033 us 26,721 us 19,204 us +31.1% +82.4%
bs=100 sw=10 sl=64 throughput 1,104 tuples/sec 1,249 tuples/sec 1,096 tuples/sec -11.6% +0.7%
bs=100 sw=10 sl=64 MB/s 0.674 MB/s 0.763 MB/s 0.669 MB/s -11.7% +0.7%
bs=100 sw=10 sl=64 p50 84,259 us 80,267 us 94,033 us +5.0% -10.4%
bs=100 sw=10 sl=64 p95 120,045 us 89,715 us 99,939 us +33.8% +20.1%
bs=100 sw=10 sl=64 p99 120,045 us 89,715 us 110,148 us +33.8% +9.0%
bs=1000 sw=10 sl=64 throughput 1,379 tuples/sec 1,411 tuples/sec 1,128 tuples/sec -2.3% +22.3%
bs=1000 sw=10 sl=64 MB/s 0.842 MB/s 0.861 MB/s 0.688 MB/s -2.2% +22.3%
bs=1000 sw=10 sl=64 p50 722,658 us 704,548 us 921,191 us +2.6% -21.6%
bs=1000 sw=10 sl=64 p95 783,066 us 784,331 us 959,735 us -0.2% -18.4%
bs=1000 sw=10 sl=64 p99 783,066 us 784,331 us 988,604 us -0.2% -20.8%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,442.35,200,128000,452,0.276,20129.49,35033.47,35033.47
1,100,10,64,20,1811.82,2000,1280000,1104,0.674,84259.45,120045.01,120045.01
2,1000,10,64,20,14504.66,20000,12800000,1379,0.842,722657.63,783065.80,783065.80

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.23744% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.76%. Comparing base (f9b899a) to head (dbcf2ec).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
...shboard/user/workflow/WorkflowPublishService.scala 84.61% 4 Missing and 10 partials ⚠️
...ra/web/resource/dashboard/SearchQueryBuilder.scala 50.00% 1 Missing ⚠️
...esource/dashboard/WorkflowSearchQueryBuilder.scala 97.82% 1 Missing ⚠️
...rce/dashboard/user/workflow/WorkflowResource.scala 98.30% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8576      +/-   ##
============================================
- Coverage     92.91%   92.76%   -0.15%     
- Complexity     4884     4908      +24     
============================================
  Files          1224     1226       +2     
  Lines         51298    51600     +302     
  Branches       6322     6368      +46     
============================================
+ Hits          47664    47869     +205     
- Misses         2070     2158      +88     
- Partials       1564     1573       +9     
Flag Coverage Δ *Carryforward flag
access-control-service 71.78% <ø> (ø) Carriedforward from bf743fb
agent-service 99.32% <ø> (ø) Carriedforward from bf743fb
amber 88.65% <92.23%> (-0.36%) ⬇️
computing-unit-managing-service 54.61% <ø> (ø) Carriedforward from bf743fb
config-service 87.25% <ø> (ø) Carriedforward from bf743fb
file-service 81.53% <ø> (ø) Carriedforward from bf743fb
frontend 96.68% <ø> (ø) Carriedforward from bf743fb
notebook-migration-service 83.73% <ø> (ø) Carriedforward from bf743fb
pyamber 98.47% <ø> (ø) Carriedforward from bf743fb
workflow-compiling-service 74.09% <ø> (ø) Carriedforward from bf743fb

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Match search and listings against the copy on show

2 participants