feat(writer): FilterReplayFile rewrites a replay without selected entities (bucket/struct/id/property rules); inspector File > Filter Entities - #48
Open
onur-zenos wants to merge 2 commits into
Conversation
…ities; inspector File > Filter Entities - VTX::FilterReplayFile(src, dst, spec, progress): drop (blacklist) or keep (whitelist) entities by unique id glob, struct name glob or scalar property value (struct.field resolved through the file's schema; empty struct = any struct with that field). Every frame survives: the header block is copied verbatim, every chunk is re-serialized with the surviving entities (same frame ranges, same per-chunk compression, fresh xxHash64 checksums) and the footer keeps total_frames, duration and the per-frame time table; only the seek table changes. Timeline events are carried through the new SessionFooter::events (both policies) and pruned to entities that still exist. Progress callback with cancel; ReplayFilterResult reports kept/dropped, per-struct drops, event counts and sizes. ReplayFilterMatcher / GlobMatch expose the rule engine for previews. vtx_writer links vtx_reader privately; the entry point is compiled only when VTX_BUILD_READER is on. - tools/inspector: File > Filter Entities window -- struct checklist with current-frame counts, unique id globs, property rules, case-insensitive and event-pruning toggles, live preview on the current frame, worker thread with chunk progress and Cancel, result summary. - reader: GetFooter() now carries chunk_index[i].checksum on both backends (was always 0) and, for FlatBuffers, each event's entity_unique_id and location. - tests: ReplayFilterTest (both backends), ReplayFilterGlob, ReplayFilterMatcher. Docs: SDK_API.md section, CHANGELOG.
…the inspector - ReplayFilterRule::Bucket(pattern): selects every entity of the buckets whose schema name matches the glob, or of the bucket at that index when the pattern is digits only (also for buckets the schema does not name). Dropping a bucket empties it in every frame; the slot stays because bucket index is positional. A bucket rule that matches no schema bucket is an error listing the available names. ReplayFilterMatcher queries now take the bucket index (Matches / Keeps / Apply) and expose BucketMatches / BucketName; ReplayFilterResult adds dropped_by_bucket. - tools/inspector: Filter Entities gets a Buckets checklist (index, name, current-frame count) above the struct list; the outcome lists drops per bucket. - tests: DropByBucketName, KeepBucketCombinesWithOtherRules, BucketByIndexAndGlob (both backends, three-bucket schema) and BucketRulesResolveAgainstSchemaBuckets. Docs and CHANGELOG updated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewrite a
.vtxreplay into a new file without selected entities. The cut (#46) slices a frame range; this filters what is in every frame. Every frame survives, the header block is copied verbatim, every chunk is re-serialized with the surviving entities, and the footer keepstotal_frames, duration and the per-frame time table. Only the seek table changes.SDK:
VTX::FilterReplayFilesdk/include/vtx/writer/core/vtx_replay_filter.h, next toRepairReplayFile.ReplayFilterSpec:Drop(blacklist: entities matching any rule go) orKeep(whitelist: only matching entities stay); rules combine as a union.Bucket(schema bucket name glob, or the bucket index as digits),UniqueIdglob,StructNameglob,Property(struct, field, value-glob)where the struct may be empty for "any struct declaring that scalar field" (Bool/Int32/Int64/Float/Double/String). Optional ASCII case-insensitive matching. Glob =*and?.SessionFooter::events(both footer serializers) and are pruned by default to entities that still exist.ReplayFilterResultreports frames, chunks, entities seen/kept/dropped, drops per struct and per bucket, event counts, sizes, elapsed.ReplayFilterMatcher(Matches/Keeps/Applyper bucket index,BucketMatches,BucketName) andGlobMatchexpose the rule engine for previews on in-memory frames.vtx_writernow linksvtx_readerprivately; the entry point is compiled only whenVTX_BUILD_READERis on.Inspector: File > Filter Entities
Buckets and structs checklists with current-frame counts, unique id globs (one per line), property rules (struct / field / value glob), case-insensitive and event-pruning toggles. A live preview shows what the rules keep per bucket on the current frame and reports unresolvable rules before anything runs. The rewrite runs on a worker thread with a chunk progress bar and Cancel, then reports frames, kept/dropped entities, per-bucket and per-struct drops, event counts and sizes.
Reader fix found on the way
GetFooter().chunk_index[i].checksumwas always 0 on both backends: the footer-to-native conversions copied every seek-table field except the checksum (the internal seek table used for chunk loading had it). Visible aschecksum: 0for every chunk in the CLI'schunkscommand; the cut copied 0 into verbatim entries. The FlatBuffers footer conversion also dropped each event'sentity_unique_idandlocation. All three conversions now carry the fields.Tests and verification
tests/writer/test_replay_filter.cpp: 27 tests.ReplayFilterTest.*on both backends (struct / unique id / property / numeric / case-insensitive / bucket-by-name / bucket-by-index-and-glob / keep-bucket rules, keep-nothing, events carried and pruned, error paths, cancel) plusReplayFilterGlobandReplayFilterMatcherunit tests. Full suite green (456 tests, Windows Release static, samples on).Docs: new "Filtering entities into a new replay" section in
docs/SDK_API.md; CHANGELOG under Unreleased.