Skip to content

fix(search): reserve interactive embeddings from their own admission lane - #7726

Merged
waleedlatif1 merged 4 commits into
stagingfrom
fix/interactive-embedding-lane
Sep 10, 2026
Merged

waleedlatif1 merged 4 commits into
stagingfrom
fix/interactive-embedding-lane

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Search queries embedded through the same per-credential admission bucket as bulk indexing, so during a crawl one interactive request competed with hundreds of batches
  • Every embedding call still reserves from the credential's aggregate request and token buckets, so the configured budget is never exceeded. Checkpointed (bulk indexing) calls additionally reserve from bulk buckets capped at 90% of that budget, which leaves interactive callers guaranteed headroom
  • Cooldown and quota gates stay shared, so a provider 429 or exhausted balance still pauses every caller

Type of Change

  • Bug fix

Testing

  • Admission tests assert bulk calls reserve aggregate plus 90% bulk buckets with unchanged cooldown keys, a bulk batch above the lane cap is rejected up front, and a one-per-minute budget still leaves one bulk request slot; client test asserts checkpointed calls are bulk and plain calls are not
  • vitest on lib/embeddings, lib/core/rate-limiter, lib/knowledge/embeddings: 342 passing; type-check and lint pass

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

…lane

A search embeds its query through the same per-credential admission bucket
as bulk indexing, so during a crawl one short interactive request competed
with hundreds of batches for the bucket's refill. Non-checkpointed callers
now reserve from a separate lane under the same identity; the provider
cooldown and quota gates stay shared, so a provider 429 or an exhausted
balance still pauses every caller.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBacX6HGVhPMUySuMfANwn
@vercel

vercel Bot commented Sep 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 10, 2026 9:49am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; the current implementation fixes the previously reported aggregate-budget and undersized-bucket failures without introducing a new actionable defect.

Summary

  • Bulk calls reserve from both aggregate buckets and bulk buckets capped at 90% of configured capacity.
  • Oversized bulk token reservations are rejected before admission, while minimum request budgets retain one usable bulk slot.
  • Embedding client tests verify checkpointed calls are classified as bulk and ordinary calls remain interactive.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Embedding request] --> B[Reserve aggregate credential buckets]
    B --> C{Checkpointed bulk call?}
    C -->|No| D[Proceed using aggregate capacity]
    C -->|Yes| E[Reserve 90% bulk buckets]
    E --> F{Bulk token cost fits lane cap?}
    F -->|No| G[Reject before waiting]
    F -->|Yes| H[Proceed when aggregate and bulk capacity are available]
    D --> I[Shared cooldown and quota gates]
    H --> I
Loading

Reviews (3) · Last reviewed commit: "refactor(search): derive the bulk admiss..."

Comment thread apps/sim/lib/core/rate-limiter/provider-admission.ts Outdated
Every caller reserves from the credential's aggregate buckets, and the bulk
lane additionally reserves from a bucket capped at 90% of that budget. The
aggregate can no longer exceed the configured budget, and interactive callers
always find headroom instead of a queue behind a crawl's batches.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBacX6HGVhPMUySuMfANwn
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile review

Comment thread apps/sim/lib/core/rate-limiter/provider-admission.ts Outdated
waleedlatif1 and others added 2 commits September 10, 2026 02:45
…vation

At the smallest supported budgets the 90% share floored below a single
request or token cost and locked the bulk lane. The bucket now holds at least
the reservation it is asked for, while the refill rate keeps the share.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBacX6HGVhPMUySuMfANwn
A single bulk flag now selects the shorter admission wait and the capped
lane, the lane cap is a pure function of configuration, and a bulk batch
larger than that cap is rejected up front instead of resizing a shared
bucket. Tests use the shared env mock.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile review

@waleedlatif1
waleedlatif1 merged commit bba8f90 into staging Sep 10, 2026
32 of 33 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/interactive-embedding-lane branch September 10, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant