[embrace897]

PR Checklist

• The proposed commit message follows Conventional Commits.

Bugs / Features

• Related issue(s) linked. Reproduced directly on main.
• Tests for the changes have been added.
• Docs have been added or updated.
• A patch changeset for @voltagent/rag has been added.

What is the current behavior?

When the parser returns no blocks and a fenced block exceeds maxTokens, CodeChunker returns token offsets relative to the fence body. With prose before the fence, using text.slice(chunk.start, chunk.end) can return prose while chunk.content contains code. The line and column metadata is also relative to the fence body.

What is the new behavior?

Translate the split chunks' start and end offsets to the original document and use those offsets to compute line and column metadata. Regression tests exercise LF and CRLF documents with prose before the fence.

Notes for reviewers

• Both new regression cases fail on upstream main and pass with this patch.
• vitest run --typecheck passed all 49 RAG tests in 20 files.
• Scoped strict TypeScript, changed-file Biome, changeset Prettier and whitespace checks passed.
• CommonJS, ESM and type declaration builds passed.

OpenAI Codex assisted with reproduction, implementation and local validation.

---

Summary by cubic

Fixes CodeChunker reporting token offsets relative to the fence body when a fenced code block exceeds maxTokens and is split by the fallback. Offsets and line/column metadata now point to the original document, so text.slice(chunk.start, chunk.end) returns the code content instead of possibly prose before the fence. Adds regression tests for LF and CRLF documents with prose before the fence.

^{Written for commit 68fb472. Summary will update on new commits.}

Summary by CodeRabbit

• Bug Fixes
  • Corrected position reporting for code chunks created when oversized code blocks are split as a fallback. Chunk offsets now point to the corresponding text in the original document, with line and column positions consistent across LF and CRLF line endings. This makes it easier to locate each chunk’s content in the source document.

[changeset-bot]

🦋 Changeset detected

Latest commit: 68fb472

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@voltagent/rag	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[coderabbitai]

No actionable comments were generated in the recent review.

Walkthrough

Token-fallback chunks from fenced code now report offsets and metadata positions relative to the full input document. Tests cover LF and CRLF input when the parser returns no results. A Changeset declares a patch release for @voltagent/rag.

Changes: Fenced code offset reporting

Layer / File(s)	Summary
Original-document offset reporting packages/rag/src/chunkers/code-chunker.ts, packages/rag/src/chunkers/code-chunker.spec.ts, .changeset/fenced-code-document-offsets.md	Token-fallback chunk offsets and metadata positions now include the code segment’s starting offset. Tests check chunks, source slices, and line and column positions for LF and CRLF input. The Changeset declares a patch release for @voltagent/rag.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: Minimal · up to 68fb4

Split fenced-code chunks now report offsets relative to the original document, so slicing the document returns the chunk's code. The change is small and tested, and no merge-blocking risk remains.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title names the fix in the rag package: CodeChunker reports original-document offsets for split fenced code. This matches the main change in the code-chunker.ts summary and the PR objectives, so t…
Description check		The description follows the template. It fills in the checklist, the current and new behavior sections, and reviewer notes. It leaves the 'Related issue(s) linked' and 'Docs have been added or updated…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Title check

Explanation

The title names the fix in the rag package: CodeChunker reports original-document offsets for split fenced code. This matches the main change in the code-chunker.ts summary and the PR objectives, so the title is related to the main change and specific enough to scan.

Full details: Description check

Explanation

The description follows the template. It fills in the checklist, the current and new behavior sections, and reviewer notes. It leaves the 'Related issue(s) linked' and 'Docs have been added or updated' items unchecked and gives no 'fixes (issue)' reference. The changeset and test items are checked, and the behavior is explained clearly, so the description is mostly complete.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

Recent review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: d2c8bc4e-53cc-4da9-ae1e-2337739d944d

Commits

Reviewing files that changed from the base of the PR and between e9fc599 and 68fb472.

Files selected for processing: 3

• .changeset/fenced-code-document-offsets.md
• packages/rag/src/chunkers/code-chunker.spec.ts
• packages/rag/src/chunkers/code-chunker.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

[cubic-dev-ai]

No issues found across 3 files

_{View guided diff | Re-trigger cubic}