Conversation
Cache table-model source matches by database and table name instead of device id.\n\nMatch each table once per event and keep table-model TsFile table names complete for privilege checks.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18648 +/- ##
============================================
- Coverage 42.82% 42.82% -0.01%
Complexity 442 442
============================================
Files 5451 5451
Lines 395425 395430 +5
Branches 51805 51808 +3
============================================
+ Hits 169349 169351 +2
- Misses 226076 226079 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Caideyipi
left a comment
There was a problem hiding this comment.
Thanks for this optimization. The table-level cache is a useful direction. I left two inline comments, including one authorization-cache invalidation race that can affect correctness.
Could you add focused coverage for the changed matching behavior as well?
- Multiple devices from one table should invoke table matching only once.
- A multi-table TsFile should still record every table after all sources have matched.
- Ideally, a cache-invalidation/concurrent-refill test should demonstrate that stale authorization results cannot survive an invalidation.
| // Use full cache to avoid queue stuck and block insertion | ||
| protected final Map<IDeviceID, Set<PipeRealtimeDataRegionSource>> deviceToSourcesCache; | ||
| protected final Map<Pair<String, IDeviceID>, Set<PipeRealtimeDataRegionSource>> | ||
| protected final Map<Pair<String, String>, Set<PipeRealtimeDataRegionSource>> |
There was a problem hiding this comment.
[P2] Prevent stale authorization results from refilling this cache
AuthorityChecker.invalidateCache() currently clears this matcher before invalidating the authority cache. That leaves a stale-refill window: a concurrent event can miss this cache, read the old authorization result, and cache a table-wide denial here. After the grant invalidation completes, later events can keep hitting that stale entry, remain unmatched, and advance progress until another invalidation or source change.
Please invalidate the authority cache before this matcher (also in invalidateAllCache()), or coordinate the two caches with a generation/version so that an entry computed before invalidation cannot be installed afterward.
| return new Pair<>(matchedSources, findUnmatchedSources(matchedSources)); | ||
| } | ||
|
|
||
| final String tableModelDatabaseName = |
There was a problem hiding this comment.
[P3] Resolve the table-model database name only for table events
getTableModelDatabaseName() is evaluated before the model check, so every insertion event, including tree-model events, now performs its lazy substring/toLowerCase initialization on the realtime write path even though the value is only used for table-model matching below. Please move this lookup into the table-model branch.
Invalidate the authority cache before the pipe matcher cache to avoid stale authorization refills.\n\nResolve the table-model database name only for table events and add matcher orchestration coverage.
|
Addressed in 77731ba.
The testCachedMatcher performance test is unchanged from master; its runtime is pre-existing. |
Summary
Validation