Skip to content

fix: paginate filesystem key-value stores by logical key - #2431

Open
peterolkhov wants to merge 1 commit into
apify:masterfrom
peterolkhov:codex/fix-filesystem-kvs-cursor
Open

peterolkhov wants to merge 1 commit into
apify:masterfrom
peterolkhov:codex/fix-filesystem-kvs-cursor

Conversation

@peterolkhov

Copy link
Copy Markdown

Description

Filesystem key-value stores sort physical filenames but return logical metadata keys. With item and item-1, one-record pages omit item; a filename alias such as the SDK's INPUT.json can return INPUT again after that cursor.

This uses validated metadata keys for sorting and cursor filtering before applying the limit. It adds native pagination, metadata-alias, and invalid-metadata coverage. Filenames, stored values, public signatures, and log messages stay unchanged.

Listing now reads metadata for every record before yielding a page. It keeps metadata in memory for sorting, without reading record values.

AI assistance: I used Codex to investigate, implement, and test this change.

Issues

No linked issue.

Testing

  • uv run poe lint: passed.
  • uv run ty check for the changed source and test files: passed.
  • Native storage unit suites: 1,082 passed, 6 skipped.
  • Native local HTTP storage integration tests: 3 passed.
  • Real Apify SDK replay: bare INPUT and INPUT.json both end after cursor INPUT.
  • uv run poe check-code was attempted. Full type checking is blocked by missing optional dependencies and a type error in the unchanged BeautifulSoup parser. Full unit and integration attempts stop during collection on optional dependency imports. Hosted CI has not run.

Checklist

  • CI passed

@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Nov 9, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant