[MP][Maru] Maru CXL shared L1 backend for MP mode - #20
Draft
seohui-XCENA wants to merge 65 commits into
Draft
seohui-XCENA wants to merge 65 commits into
seohui-XCENA wants to merge 65 commits into
Conversation
seohui-XCENA
force-pushed
the
maru-mp-l1
branch
3 times, most recently
from
July 3, 2026 10:19
2477191 to
ce54689
Compare
jooho-XCENA
reviewed
Jul 6, 2026
jooho-XCENA
left a comment
jooho-XCENA
left a comment
There was a problem hiding this comment.
docs/coding_standards.md 기준 리뷰입니다. error 항목만 코멘트로 답니다 (warning/info는 별도 공유).
[error] 신규 maru L1 백엔드에 in-repo 설계 문서가 없고, PR 설명이 인용하는 설계 정본이 dangling 참조임
PR 설명은 temp_docs/mp/design/maru_l1_manager.md · temp_docs/mp/design/maru_l1_api_mapping.md를 설계 정본으로 인용하지만, temp_docs/는 PR 트리·브랜치 히스토리 어디에도 없습니다 (git ls-tree -r HEAD | grep temp_docs → 없음). 이 PR은 maru_l1_manager.py(1,057줄) · maru_memory_allocator.py(243줄) · l1_protocol.py(113줄)를 추가하면서 docs/를 건드리지 않아, coding_standards §5.1(non-trivial 신규 기능은 docs/design/<path>/ 설계 문서 필수)에 걸리고 리뷰어가 설계 정합 검토(리뷰 Step 2)를 할 수 없습니다.
수정 제안: 설계 정본을 docs/design/v1/distributed/(예: maru_l1.md — MaruL1Manager / MaruMemoryAllocator / L1ManagerInterface seam / orphan sweeper·crash-recovery 전제 포함)로 이 PR에 추가하고, PR 설명의 참조를 in-repo 경로로 갱신해 주세요.
kihwan-XCENA
reviewed
Jul 7, 2026
kihwan-XCENA
left a comment
kihwan-XCENA
left a comment
There was a problem hiding this comment.
주호님이 위에 리뷰로 언급하신 것 처럼 _pending_write와 _pending_read가 동시에 존재 할 수 있는 상황이 되어도 괜찮을까요?
하나의 KV Cache가 _pending_write 및 _pending_read에 동시에 저장되는 경우가 있는 것 같습니다.
Author
|
[error] 설계 문서 (in-repo) — 수정했습니다 (
|
Author
|
@kihwan-XCENA 좋은 지적 감사합니다 — 실제로 가능한 버그였습니다(주호님 리뷰의 mid-write reserve_read 건과 동일 뿌리).
|
- l1_protocol.py: structural runtime_checkable Protocol mirroring L1Manager's 17-method surface, with per-method listener/lock contract docstrings - config.py: MaruL1Config + maru_config field, __post_init__ DRAM-clamp skip, maru CLI args (--maru-server-url/--maru-pool-size-gb/--maru-instance-id), parse_args_to_config wiring - tests: interface<->L1Manager method-set + signature conformance, maru config parsing Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- thin wrapper over external maru_lmcache.CxlMemoryAdapter, lazy-imported - two-phase init_layout: build MaruHandler + CxlMemoryAdapter on first layout (single-model; layout mismatch rejected) - free/batched_free no-op (page lifecycle owned by MaruServer); abort_alloc discards an allocated-but-unregistered page - MaruL1Config -> maru.MaruConfig mapping; maru-only get_by_location / create_store_handle / handler surface for MaruL1Manager - tests use mocked maru runtime (no CXL required) Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
…ifecycle) - sibling of L1Manager over the maru shared CXL pool: membership/read protection live in the MaruServer directory (pin_count), locally only in-flight staging (_pending_read refcount, _pending_write) - reserve_read: per-key independent pins (1+extra_count via one RPC), rollback on partial pin / retrieve failure / unresolvable page - reserve_write mode=new: local staged check + batch_exists dedup (cross-instance), all-or-nothing OOM; finish_write: batch_store, dup-skip is success, definitive False reclaims the page, unknown server state never recycles - delete: staged keys and pinned keys refuse with KEY_IS_LOCKED (exists() disambiguates the handler's pinned/missing conflation) - clear(force=False) keeps locked staging (stock parity); close drains - PARITY/MARU provenance comments; RPC reply length guards - tests: stateful fake maru runtime with fault-injection knobs, failure-path coverage, and a conformance suite parametrized over both L1Manager (CUDA) and MaruL1Manager Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- reserve_read/reserve_write/finish_write/finish_read/delete/clear now fire the on_l1_keys_* events (feeds the eviction LRU and the store controller), reporting only the keys that actually succeeded - finish_read: temporary reads (local staging, never directory-pinned) are reclaimed via abort_alloc at refcount zero and fire deleted_by_manager; normal reads still unpin -- _drain_staging mirrors the same branch - touch_keys: no-op -> fires on_l1_keys_accessed (unsynchronized, like stock) - event_bus (mp_observability) publish deferred: observability-only, no tiering-control impact - tests: RecordingListener fake; maru firing edge cases (hit-only, store-fail, temporary reclaim, removed-only, clear) + cross-backend conformance lifecycle test parametrized over stock and maru Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- promote branches on the staged is_temporary flag (Decision A): temporary (default prefetch) moves the loaded page straight to read staging with no batch_store and no pin -- finish_read reclaims it at refcount zero; retained (prefetch_policy: retain) registers via batch_store then re-resolves the authoritative page with pins so a dup-skip that auto-freed our page still yields the winning shared page - fires on_l1_keys_finish_write_and_reserve_read, never on_l1_keys_write_finished (the latter would make the store controller re-store the key to L2) - extract _pin_retrieve_stage (shared by reserve_read + retained promote) and _store_staged (shared by finish_write + retained promote); reserve_read and finish_write refactored onto them, behavior unchanged - document the load-failure cleanup gap (finish_write->delete on failed keys publishes then removes; caller-side batch-abort is the future fix) - tests: temporary/retained promote, dup-skip re-resolve, extra_count pins, wrong-state/unstaged guards, store-failure, anti re-store event check; cross-backend conformance for retained promote, temporary drop, promote event Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- daemon sweeper (started in __init__, stopped in close) scans staging under the manager lock and reclaims entries whose TTL has elapsed: an expired write page is returned to the owner (abort_alloc); an expired read releases its pins (a temporary read reclaims its private page instead) - _PendingRead/_PendingWrite carry a monotonic deadline (default never); set at reserve/promote and refreshed on overlapping reserve (mirrors a stock re-lock extending the TTL). abandonment is a time judgement -- a refcount says how many holds exist, not whether they will ever be released - no listener fires on sweep: a late finish_read/unsafe_read then sees KEY_NOT_EXIST and recomputes (same failure path as a stock TTL expiry), and firing across the daemon thread would be a novel hazard for stock listeners - tests: sweeper lifecycle, expired write/read/temporary reclaim, live staging left intact, overlapping reserve refreshes the deadline, late finish is KEY_NOT_EXIST Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- store/prefetch/eviction controllers type their l1_manager param as the structural L1ManagerInterface instead of the concrete L1Manager, so either L1Manager or MaruL1Manager can drive them; runtime behavior unchanged - the controllers call only Protocol methods (reserve/finish read+write, finish_write_and_reserve_read, delete, is_key_evictable, get_memory_usage, register_listener), so L1Manager still satisfies the param structurally Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
…_layout - StorageManager.__init__ selects the L1 backend: MaruL1Manager when memory_config.maru_config is set, else the stock L1Manager; typed as the shared L1ManagerInterface so the controllers drive either unchanged - new StorageManager.register_kv_layout: maru-gated (isinstance) forward that brings up the CXL pool once the layout is known, rejecting >1 object group (single-model maru limit); a no-op for stock. Engine call site lands later - get_l1_memory_desc is now L1MemoryDesc | None (maru has no single registerable region); the l1_memory_desc property raises if accessed while None (its only consumer, p2p, is rejected at startup for maru) - widen SerdeL2AdapterWrapper l1_manager param to L1ManagerInterface (like the controllers); collect its reserve_write buffers in one pass so the success path types as list[MemoryObj] - tests: maru vs stock selection, register_kv_layout forward / multi-group reject / stock no-op Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
…/p2p/engine) - validate_storage_manager_config: a maru L1 config rejects the other L1 backends (gds-l1-path, l1-devdax-path), store_policy=skip_l1, and L2 adapters that require a single registerable region (nixl / mooncake-rdma); copy-type L2 and the default store policy pass - l1_exposes_single_memory_region returns False for maru (the shared CXL pool has no single registerable region), so the existing p2p startup guard fires; its message now lists maru too - run_http_server rejects maru paired with a non-lmcache_driven transfer mode (engine-driven/auto assume engine-side buffers the shared pool lacks) - tests: maru + gds/devdax/skip_l1/registered-L2 raise; maru + copy-L2 and the default policy pass; l1_exposes False for maru Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
…ol bring-up - after the KV layout is resolved in register_kv_cache, call StorageManager.register_kv_layout with the layout desc, the storage MemoryFormat (KV_MLA_FMT / KV_2LTD via is_mla), chunk size, and object-group count; a no-op for the stock backend, idempotent for maru across instances - on failure (e.g. maru rejecting >1 object group) close the just-built cache context and re-raise so the instance is not left half-registered - runtime test deferred: importing this module needs a c_ops rebuild (built .so lacks execute_object_group_transfer); lands with the C9 G6 test post-rebuild Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
…ister_kv_layout) - G6 (C9): run_http_server rejects maru paired with a non-lmcache_driven transfer mode - C10: register_kv_cache forwards to StorageManager.register_kv_layout with the right MemoryFormat (KV_MLA_FMT / KV_2LTD via is_mla); on a rejected layout it closes the cache context and leaves the instance unregistered; stock registers normally - both were blocked by a stale c_ops build (missing execute_object_group_transfer); unblocked after the rebuild Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- MaruL1Manager now publishes the same L1 events to the shared event bus as stock L1Manager, alongside the listener notifications: L1_READ_RESERVED, L1_READ_FINISHED (+ L1_KEYS_EVICTED for freed temporaries), L1_WRITE_RESERVED, L1_WRITE_FINISHED, L1_WRITE_FINISHED_AND_READ_RESERVED, and L1_KEYS_EVICTED for delete/clear; touch_keys stays listener-only (matches stock) - add get_event_bus() in __init__ and a _publish helper mirroring stock - closes the observability gap deferred in C4 -- maru L1 ops now surface in the mp_observability dashboards - tests: write/read/delete lifecycle events, promote publishes the promote event (never L1_WRITE_FINISHED), temporary finish_read publishes evicted, touch_keys publishes nothing Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
…stack - a StorageManager harness backed by the maru fakes (fake CXL pool + MaruServer directory) + a mock L2, driving the real Store/Prefetch/Eviction controllers over MaruL1Manager - covers store -> maru directory registration, prefetch of directory-resident keys as a full L1 hit (maru reserve_read), and watermark eviction driving MaruL1Manager.delete on the shared directory - L1<->L2 byte movement (write-through, promote) is stock controller logic that needs real-memory-backed pages and is covered by the stock StorageManager tests, so it is not re-asserted here; extra_count and cross-node PINNED are covered at the manager level (C3/C5) Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
…acts) - add cross-backend conformance cases for the shared error paths: reserve_read of a mid-write key is KEY_NOT_READABLE (not a miss), finish_read/finish_write on an unstaged key is KEY_NOT_EXIST, delete of a missing key is KEY_NOT_EXIST - clear(force=True) is intentionally not conformance-tested: stock clears all objects while maru only drops local staging (the shared directory belongs to other instances) -- a legitimate divergence, not drift Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- get_memory_usage total = owned pool + CXL device free (cxl_pool.free_size from get_stats), so the eviction watermark tracks whole-device fill instead of just this instance's owned pool (which evicts prematurely while the device still has room) - cache the last-known free and reuse it when a get_stats RPC omits cxl_pool (transient timeout / older server) so total never momentarily collapses to the owned pool and fires a spurious eviction Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- Track server pins on _PendingRead.pinned separately from refcount, so a temporary stage that absorbs an overlapping reserve_read's pins releases them; previously those pins leaked and left the page un-evictable on MaruServer. - Exclude mid-write keys from reserve_read's pin/stage step: a key mid-write on this instance stays KEY_NOT_READABLE even when a peer has registered it, instead of being double-staged in both _pending_write and _pending_read (which stranded the in-flight write and failed its promote). - Add regression tests for both paths. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- Add docstrings to allocate, batched_allocate, close, and memcheck. - Note the RuntimeError raised when called before init_layout(), and the out-of-memory -> None contract that reserve_write's all-or-nothing handling depends on. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- Add docs/design/v1/distributed/maru_l1.md: the sibling + Protocol seam, the MaruL1Manager API contracts, the pending-read/pending-write state-machine invariants, the TTL sweeper, and the crash-recovery premises. - Document the Maru CXL shared L1 tier in the MP configuration guide and the architecture index; point the deprecated in-process maru page to it. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- Add MaruL1Config.auto_expand (default True) with a --maru-auto-expand / --no-maru-auto-expand CLI flag, threaded into MaruConfig via the allocator. - Branch get_memory_usage on it: auto_expand on keeps the device-fill watermark (owned pool + CXL device free); off anchors total to the owned pool, so a hard-capped pool evicts before it is exhausted instead of OOMing while the device still has free space the pool cannot grow into. - Extend the fake handler with a cxl_free knob and add a watermark test. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- Rewrite docs/design/v1/distributed/maru_l1.md to the l2_adapters house style (Overview + components + operation flow + configuration + limits); document the auto_expand knob and the single-device region bound, and drop the internals-heavy framing. - Correct MaruMemoryAllocator docstrings: page reclamation is driven by LMCache eviction via MaruL1Manager (delete / abort_alloc), not the allocator, so free/batched_free are no-ops. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
- Merge the maru-specific test files into one unit file (test_maru_l1_manager.py): fold in the allocator, config/startup-guard, and control-integration tests, and drop cases already covered by the shared L1Manager conformance suite (notably the notification tests: ~14 -> 4). - Absorb test_l1_protocol.py (structural conformance + shared helpers) into test_l1_manager_conformance.py. - Delete the now-empty test_maru_memory_allocator.py, test_l1_config_maru.py, test_maru_integration.py, and test_l1_protocol.py. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
ead76c8 ("Merge upstream LMCache dev") landed unresolved, breaking every CI job. Three separate defects, all from that merge: - conflict markers were committed into run_http_server, so the module failed to parse: ruff/mypy syntax errors, pytest aborting collection on test_http_server.py + test_http_quota_endpoints.py (no test in the suite ran), and the lmcache CLI failing to start. Keep both guards -- the maru lmcache_driven transfer-mode check and the upstream coordinator event-reporting check -- and document the former in the Raises section - the is_mla import in maru_l1_manager.py was left out of isort order - upstream API moves were not carried into the maru tests: PrefetchRequestSpec now takes group_layout_descs (dict) instead of layout_desc, and EngineKVFormat moved from lmcache.c_ops to lmcache.lmcache_native. The two register_kv_layout annotations were dangling on lmc_ops as well; string annotations, so mypy never flagged them Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
Signed-off-by: baoloongmao <307499405@qq.com>
… block (LMCache#4787) Signed-off-by: Meghana <meghana.madhyastha@parasail.io> Co-authored-by: maobaolong <baoloongmao@tencent.com>
feat(blend): rope-less engine-group sentinel (-1) in group_to_cache Signed-off-by: deng451e <838677410@qq.com>
…or.py to %-format (LMCache#5093) (LMCache#5094) [good-first-issue] storage: convert f-string log calls in eic_connector.py to %-format Convert all 51 f-string logger calls to lazy %-style formatting. This also fixes six log messages whose unprefixed continuation literals rendered placeholders such as "{err_code}" verbatim. Signed-off-by: Yifan Jin <53075473+chrisyifanjin@users.noreply.github.com>
Preserve Maru selection and GDS options, adopt total read-lock counts, and report Maru-owned configured capacity. Add regression coverage for count balancing and capacity reporting.
refactor(mp): encapsulate context unregistration Signed-off-by: baoloongmao <307499405@qq.com>
…5088) fix(cli): avoid hanging on bench ZMQ shutdown Signed-off-by: nautaa <870284156@qq.com>
…che#3510) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Refactor IPC policy into platform layer Signed-off-by: baoloongmao <307499405@qq.com>
…he#5057) Signed-off-by: Clouddude <klouddude@gmail.com> Signed-off-by: Hyunseung Jung <91711838+okhs0712@users.noreply.github.com> Co-authored-by: Clouddude <klouddude@gmail.com>
Signed-off-by: Sangyoon Kwon <syk0905.kwon@samsung.com>
…he#4847) * [Core] Add eviction-aware lazy offload policy Buffers store operations on the scheduler and releases them when the GPU blocks holding their data approach the free queue's eviction head, or when they pass the configured deferral deadline. Operations whose blocks are recycled before they come due are dropped and counted, never stored stale. EVICTION_AWARE becomes the default policy. FIFO stays available as an explicit legacy fallback and is reworked to take its eligibility inputs (finished and blocked request ids) from the controller instead of tracking request lifecycle itself. Both policies implement one OffloadPolicy interface, so policy selection is a factory rather than a wrapper that branches on the configured mode in every method; LazyOffloadPendingStore, whose remaining job was that dispatch, is removed and its tests are adapted to the new interface. The policy module is pure: vLLM types appear only in annotations and it decides without acting, so it is unit-testable without a GPU. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * [Core] Wire lazy offload through the MP connector and worker adapter Scheduler-side lazy-offload integration moves behind LazyOffloadManager. The connector forwards vLLM lifecycle events and applies the returned actions, so a policy change adds no policy-specific branches to it. The manager owns policy construction, block pin/unpin, store-batch coalescing, completion handling, and deferred session release. LazyOffloadRequestRegistry gives each request id an explicit store epoch and at most one submitted batch, so a preemption reset or a reused finished id cannot let a stale receipt tear down the successor's session. The worker adapter now reports failed stores alongside completion receipts, letting the scheduler break a request's stored-prefix chain before considering its later chunks. Every submit produces exactly one receipt per rank, including the non-writer and unhealthy paths, so pinned blocks are always released. Lazy offload now requires vLLM prefix caching: eviction detection reads block hashes, which only exist while prefix caching maintains them. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * Docs: lazy offload design and configuration reference Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * Cut lazy-offload docstrings and comments to repo density Docstring lines across the new modules go from 633 to 365, and comment lines from 109 to 84, putting the doc+comment/code ratio at 0.44 against 0.63 and 0.59 for lmcache_mp_connector.py and vllm_multi_process_adapter.py. The config class no longer repeats the per-knob tuning guidance already in docs/source/mp/configuration.rst, Args sections that only restated the signature are gone, and single-clause comments moved onto their code line. Also drops the design doc's description of a throttle WARNING that was removed with the earlier slimming pass. No behaviour change. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * Restore OffloadPolicy as an ABC and drop BlockPoolReader Upstream declared OffloadPolicy as an ABC with abstract methods. Bringing it back as a Protocol left neither implementation declaring the interface, so drift was caught by mypy alone; both are ours and in the same package, so the ABC is the right shape. FIFOOffloadPolicy and EvictionAwareStoreQueue now inherit it. BlockPoolReader existed so tests could fake the pool, but the eviction-aware tests are not in this PR: it had one implementation and no fakes. The queue takes GPUBlockPoolView directly. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * Trim policy observability and rewrite the eviction-aware design doc Drops the three profiling counters (free_queue_blocks_read, requests_validated, blocks_validated) that measured the decision loop's own cost on a dev branch. Removing them also removes _COST_SENSOR_FIELDS and shrinks the ledger change test to skipping drain_steps. The drop-sample helper folds into its one call site and _log_drain into drain(). stats() and num_pending_ops() had no caller left once the eviction-aware tests moved out. The ledger equation and every counter it names are unchanged. The design doc described stats(), the deleted counters, the removed pending store facade, a per-request block reference index, an allocated_block_ids=None test path, and a test file that is not in this PR. Rewritten against the current code at 191 lines from 237. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * Move the last lazy-offload test to the development branch Leaves this PR with no tests, per the review request to keep only the production change here. The file is archived at records/2026/09/01/artifacts/pr_tests/ on lazy_offloading_policy_dev, where its README says how to restore it; it passed 12/12 at dcfc59c. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * Flatten the eviction-aware drain to a single full-scan pass Validate every unblocked pending request each drain instead of keeping a block-to-request reverse index and an allocation-touched set; pending depth is bounded by concurrent requests, so the index cost more code than it saved. Walk the free queue once to the danger depth and drop the within-drain pin cascade widening: a candidate shifted into danger is caught one step later, inside the horizon margin. Read the block pool directly instead of through GPUBlockPoolView, build the drain output without the DrainResult intermediate, and dissolve the _PendingOperations holder into a plain dict. Drop the pure sensor counters (deferral drains, evicted tokens, throttled drains, drain_steps); the ledger still closes and emitted_overdue stays as the deadline's evidence. The periodic ledger line stays: shutdown logging is best-effort under SIGINT. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * [Core] Address review: drop epoch, document interfaces, fix deferral clock The epoch counter existed to answer one question: did this store receipt belong to the request generation that is running now? Only the failure path asked it. A preemption reset or an id reuse leaves a batch in flight that no longer belongs to the id's current holder, and its failure must not drop that holder's fresh buffered ops or blacklist its prefix. Replace the counter with a flag on the record that already exists: SubmittedStoreBatch.orphaned, set by reset() and by finished-id reuse in arrive(). The failure guard reads the flag. Everything else the epoch touched was a defensive assertion of a fact that the synchronous reset already guarantees, so PendingStoreOp.epoch, PendingStoreItem.epoch, the epoch parameter of add(), ensure_active's return value, is_current_epoch and both stale-epoch RuntimeErrors are gone. Behavior is unchanged. Also from review: an Attributes section on every dataclass the change touches, Args, Returns and Raises on the public methods, the module-level helpers and the long private helpers of the manager and both policies, a terminology block so store operation, buffered, emitted and prefix chain are defined where they are used, update/query grouping in LazyOffloadRequestRegistry, and two comment trailers trimmed in the connector. Overrides that add no behavior keep their one-line docstring, per docs/coding_standards.md 3.1. One functional fix from review: PendingStoreOp.admitted_at_time took the step clock, which drain() sets. Operations buffered before the first drain therefore carried 0.0 and, with max_deferral_seconds > 0, were overdue on sight; the same held after an idle gap, where the last drain's clock is arbitrarily old. It now reads time.monotonic() at admission. The default max_deferral_seconds of 0.0 disables the deadline, so shipped behavior was unaffected. FIFO's config reads cast to str | int | float, the union int() accepts, instead of casting to int and converting anyway. mypy needs a cast here: ConfigValue includes list[str] and None. Pre-PR pass. Ablation removed four things nothing decided on: PendingStoreOp.request_id and its two token-range fields (written at admission, read nowhere; the buffer is keyed by request id and holds each request's ops in token order), RequestSlot.awaiting_rearrival (the phase already tells a re-arrival from an id reuse: a reset leaves the slot ACTIVE, only finish() sets FINISHED), and the registry's ensure_active (every store candidate is preceded by the arrival that opens the slot, and a default slot is indistinguishable from no slot in every query). The return values of on_request_reset and discard_for_reuse were read by nobody and are now None. Docs and contracts. configuration.rst listed the lazy-offload keys twice after this change, with lazy_offload_policy defaulting to FIFO in one block and EVICTION_AWARE in the other; the superseded block is gone. The manager now states the token-contiguity precondition of the store path and the Raises of on_scheduler_step, records log_final_stats as the second method that works unbound, and describes the id-reuse producer of sessions_to_end and the ordering it requires of the caller. StoreCompletionTracker's parameters are positional-only, since the implementer names the first one req_id. lazy_offload.md no longer credits FIFO with a hash validation the manager does, and eviction_aware.md says the ledger's emitted means handed to the manager. Restored the rationale for flooring num_vllm_hit_tokens that moving the assignment had dropped. on_request_arrived no longer claims an in-flight batch carries the predecessor's session release: it does not, and it must not, because a session is keyed by request id and ending it there would end the successor's. Two more from the pre-PR pass. Drain priority. _OVERDUE_RANK sorted ahead of every free-queue rank, so a backlog of deadline-expired requests spent max_drain_per_step before a request holding the block vLLM was about to recycle got any of it, and that block's operations were dropped on the next step. The -1 existed to make an expired request due on a step whose danger depth is zero, not to outrank an imminent eviction; it is now a high sentinel, and a request that has both an in-window block and a passed deadline is ranked by the block. Missing the deadline costs latency, missing an eviction costs the data. An expired request still releases its whole buffer, not just the due front segment. Token ledger. The tracker builds a fresh copy of the request's whole token sequence for every operation it produces; the eager path handed each one to the worker and dropped it, but deferral retains them, and a running request's blocks are never in the free queue, so with the default max_deferral_seconds nothing is emitted until it finishes. A 32k request with 128 buffered operations held 128 copies, 32 MB, 2 GB across 64 such requests. Each list is a prefix of the next, so the manager keeps one ledger per request and rebinds every operation to it: 0.25 MB per request, 0.016 GB across 64. The ledger is dropped on preemption and on id reuse, so a successor cannot inherit its predecessor's tokens. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * Sort the eviction-aware imports The rename to EvictionAwarePolicyConfig left the re-export block out of alphabetical order. isort runs as its own pre-commit hook here; ruff's I rules are commented out in pyproject, so ruff check does not catch this. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * [Core] Document the deferral deadline's rank and extent separately A request past max_deferral_seconds that also holds a block in the danger window is ranked and sized by the window: it releases the due front segment, not its whole buffer. That fell out of moving _OVERDUE_RANK to 1 << 62 and reordering the branch to check the window first. The commit that made the change asserted the opposite in its message, and the design doc still described the earlier rule. Reported in review. The behavior is the right one. A block outside the window is either still referenced, where vLLM's prefix cache serves it and the store is wasted D2H, or free but deeper than the danger depth, where a later drain catches it. The case does arise: with sliding-window or chunked-local attention a cache hit is null-padded at the front, so touching it pulls a finished request's later blocks out of the free queue and leaves its earlier ones in. There the held-back operations are exactly the ones another request just took over. The deadline bullet now states order and extent as two facts, and the drain docstring says a deadline that passed while no block sits in the window. configuration.rst no longer calls max_deferral_seconds an upper bound: a step that schedules no tokens runs no drain at all, the blocked_request_ids guard skips a request with a store in flight, and max_drain_per_step truncates. All three predate this change. Also drop overdue_ids.discard(request_id), dead since the same commit: the set is rebuilt every drain and each request id appears once in the loop, so it can never remove anything. No behavior change. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> * [Core] Mark the lazy offload follow-ups with TODOs Two places the policy is knowingly coarse, both from reviewing the deferral deadline's extent. Free-queue pressure is the only retirement mechanism the drain models. A sliding-window layer retires a block on a token schedule, which is predictable but leaves no trace in the free queue, so an operation whose data is about to go waits for its deadline instead. An operation's blocks span every KV cache group, so on a hybrid model the shortest-lived group decides for all layers: one recycled sliding-window block drops the operation and the request's whole tail. Splitting the check needs per-group token ranges in LoadStoreOp. Signed-off-by: Bo Jiang <bo.jiang@temple.edu> --------- Signed-off-by: Bo Jiang <bo.jiang@temple.edu>
…test (LMCache#4953) Signed-off-by: baoloongmao <307499405@qq.com>
Finish temporary writes as local staging without publishing to the shared directory or triggering write-through. Preserve local read holds and reclaim idle temporary pages through delete, clear, close, and TTL cleanup without touching a peer copy of the same key. Add public API conformance and regression coverage for private completion, overlapping reads, peer isolation, cleanup, and expiry. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
Assert that the DRAM fixture exposes its L1 memory descriptor before constructing the NIXL transfer context. This narrows the optional protocol return type when newer dev enables type checking for the P2P integration test. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>
…LMCache#4895) Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com>
…er (LMCache#4079) Signed-off-by: Divy <divy@coralbricks.ai> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [MACA] Add Buildkite CI pipeline for MetaX MACA hardware
Adds a classic (non-K8s) Buildkite pipeline for the metax-maca queue,
matching the bare-metal pattern the main pipeline.yml itself already
uses on its own queue rather than the k3_tests/ K8s-based pattern used
by AMD/XPU/MUSA: this queue is a single dedicated host with one GPU,
so there's no multi-job scheduling benefit that would justify standing
up K3s/GPU-Operator for it.
- common-setup.sh: shared venv/install/env setup for both lanes below.
Reuses one fixed venv (recreated each run) rather than the
k3_tests/-style per-build-id venv, since this is a persistent host
with tens of GB of disk, not an ephemeral pod -- a fresh venv per
build would accumulate indefinitely. Includes a disk-space guard,
uv cache pruning, and MACA_MPS_MODE=1 (fixes a cross-process CUDA
event ordering timeout, confirmed against test_event_ipc_ordering.py).
- run-smoke-tests.sh: fast subset for PR feedback (core compute/
platform/native-extension paths), so contributors don't wait on the
full suite for every PR.
- run-unit-tests.sh: full suite on push to dev, matching the scope of
the main CUDA "Unit Tests" step. Both scripts delete the whole
workspace after running, matching that same step's own convention,
so builds don't accumulate artifacts on this queue's limited disk.
- pipeline.yml / buildkite-pipeline.yml / BK_WEB_SETUP.md: pipeline
definition and the web UI configuration doc, following the existing
convention from the AMD/XPU/MUSA/sglang lanes (including the
"Skip queued / cancel running branch builds" setting every one of
those lanes requires, for the same reason: a superseded build on a
single-agent queue only delays feedback on the newest commit).
Test scope excludes a handful of MACA-incompatible or unresolved cases,
each documented inline in run-unit-tests.sh with the evidence behind
it: two confirmed structural gaps (MACA's Triton backend lacks one FP8
conversion; two files under tests/v1/platform/ depend on NVIDIA-only
cuda.bindings for raw CUDA IPC, the same category ROCm is already
excluded from upstream), one confirmed environmental flake (a
single-GPU host timing issue, not reproducible in isolation), and a
cluster of MP-server-round-trip timeouts (mostly under
tests/v1/multiprocess/) that is deliberately parked unresolved for now
rather than root-caused.
Verified end to end on MetaX C500 hardware: both scripts run via their
real Buildkite invocation path (not an ad hoc substitute), smoke in
under 5 minutes, full suite in under an hour.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Exclude an additional flaky bench test from run-unit-tests.sh
test_server_bench.py::TestQueryChecksum::test_success failed with an
HTTP 503 under --maxfail=1 in a final validation run, but was not in
the failure list of an earlier no-maxfail run across the same tree --
i.e. it passes sometimes. Deselect it so it doesn't block --maxfail=1
runs; not independently confirmed as the same single-GPU-host
startup-timing race as test_instances_usage_e2e.py, but consistent
with it.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Exempt localhost from the host's proxy; drop two now-fixed exclusions
http_proxy/https_proxy (needed on this host for GitHub/PyPI
reachability) get honored by urllib/requests for ANY destination,
including 127.0.0.1, unless no_proxy explicitly exempts it. Several
tests spin up a real local HTTPServer and immediately query it
(tests/cli/test_describe.py, tests/cli/commands/bench/test_server_bench.py);
without the exemption, those requests get routed through the external
proxy, which returns a bare "503 Service Unavailable" for a random
localhost port it knows nothing about.
Found by tracing what looked like two unrelated flaky failures
(TestQueryChecksum::test_success, test_describe.py::TestFetchJson::
test_success) back to the same HTTPError: 503 shape. Reproduced 100%
(3/3) with the proxy set and no exemption, fixed 100% (3/3) with
no_proxy/NO_PROXY set. Also fixed TestLookupProtocol::
test_poll_prefetch_status_uses_request_id, previously deselected in
run-unit-tests.sh as an unexplained failure -- removed that deselect
now that it's root-caused and fixed rather than parked.
TestUnregisterKVCache in the same file is NOT fixed by this and stays
deselected as a separate, still-uninvestigated issue.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Trim internal investigation detail from CI comments
Comments in these files are public-facing; strip dates, reproduction
counts, and other internal-investigation narration down to just the
facts a reviewer or future maintainer actually needs (what's excluded,
why, and what to revisit). Correct the MACA_MPS_MODE=1 explanation:
it's required whenever multiple processes access the same GPU
concurrently, a MetaX platform requirement, not specific to one test.
Also drop self-deprecating framing ("first-pass", "not scientifically
tuned") from the smoke-test scope explanation in favor of a neutral
description of the split.
No functional change -- comment-only, verified via diff.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Drop explanation for the still-uninvestigated test exclusions
Remove the comment describing the multiprocess/test_server_bench/
test_key_directory/test_torch_ops exclusions -- these aren't
root-caused yet, so there isn't a real explanation to give. The
--ignore/--deselect flags stand on their own.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Harden the end-of-run workspace cleanup
Address Copilot review feedback on the sudo rm -rf step: prefer
BUILDKITE_BUILD_CHECKOUT_PATH (the actual checkout path Buildkite sets)
over $PWD, refuse to proceed if the resolved path is empty or "/", and
use -- before the path passed to rm.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Use "maca" (not "metax") as the PR trigger label
Matches this codebase's own naming (BUILD_WITH_MACA, MacaProfile,
maca_core.txt): the label tracks the software stack, not the vendor
name, closer to how xpu/musa's labels already work here. Only the
label string changes -- the "metax-maca" queue name and MetaX company
references elsewhere are unaffected.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Pin the upload-pipeline step to the metax-maca queue
Address review feedback from maobaolong: without an explicit agents
block, Buildkite's default scheduling for this step is unpredictable
on an org with multiple vendor queues -- it could land on an unrelated
agent or sit stuck waiting. XPU, MUSA, and sglang's equivalent files
already pin this the same way; only AMD's omits it.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Pin torch/triton/mcpy to versions matching this host's MACA SDK
MetaX's pip index carries multiple SDK lines simultaneously and an
unpinned install tracks whatever they published most recently. This
host's MACA SDK is 3.8.2.6; the index's latest torch/triton build now
targets 3.8.3.x, and pulling that unpinned produces an ABI mismatch at
import time:
ImportError: .../libtorch_cuda.so: undefined symbol: mcclCommWindowRegister
Pin torch directly, and override mcpy/triton to the matching version
right after installing requirements/maca_core.txt (which lists them
unpinned, correctly -- that file is the general, end-user-facing
requirements list, and different users' hosts run different SDK
versions, so it should keep resolving to whatever's current. This
pinning is CI-only, scoped to this script, not pushed into that shared
file).
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Exclude test_shm_allocator's 5GB pinned-memory test
Its allocation fails MACA's mcHostRegister with mcErrorInvalidValue on
this host.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Exclude the whole TestShmFileConnector class, not just one test
All 3 tests in the class share the same 5GB pinned-memory fixture, which
fails MACA's mcHostRegister the same way.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Add phase_timing_recorder.cu to the MACA build's source list
The CUDA and ROCm profiles already build this file; MACA's own profile
duplicates that source list rather than sharing it and was missed when
phase_timing_recorder.cu was added, leaving lmcache.cuda_ops built
without PhaseTimer's symbols on MACA.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
* [MACA] Ignore the new VMM CUDA IPC wrapper test file
Same reason as the existing cuda_ipc_wrapper/timeline_semaphore_event_ipc
exclusions: depends on NVIDIA's cuda.bindings, which MACA doesn't have.
The file's own ROCm skip doesn't cover MACA either.
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
---------
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
Co-authored-by: JianDan0212 <zhangyj0212@gmail.com>
Signed-off-by: haiqakhan2005 <haiqakhan2005@gmail.com>
…er.py to %-format (LMCache#4311) (LMCache#4319) Signed-off-by: vedjaw <vedant.jawandhia@gmail.com>
* Update MAINTAINERS.md with current orgs I think it's okay to have two orgs listed for people who maintain a relationship with their school, since that is possible. I'm guessing the accuracy in this case. And I may be missing someone else who needs to be shifted to "& Tensormesh". Signed-off-by: Karsten Wade <quaid@iquaid.org> * Update MAINTAINERS.md with someone I missed Thanks @chunxiaozheng for catching this, I'm still getting to know everyone at Tensormesh and missed this contributor's move to a new employer/organization. This is a good sign of a healthy ecosystem, when maintainers keep their role in the Open Source project and just have a different person paying them to be doing the work. Signed-off-by: Karsten Wade <quaid@iquaid.org> --------- Signed-off-by: Karsten Wade <quaid@iquaid.org>
… un-migrated code (LMCache#5118) (LMCache#5125) lint: enforce G004 (lazy %-format logging) with temporary ignores Enable ruff's G004 so an f-string in a logging call is rejected, and add per-file-ignores for the files that are not migrated to %-format yet; entries can be deleted one at a time as the migration continues. ruff matches per-file-ignores patterns with '*' crossing directory separators, so 'lmcache/*.py' would ignore the whole package subtree. Measured: a naive per-directory list also ignores 771 already-migrated files and leaves only 494 of 1337 files protected. The list is therefore generated from the actual violations: '<dir>/**' only where every file below the directory violates the rule, an explicit file entry otherwise. Signed-off-by: Runguo LI <runguo.ai@gmail.com> Co-authored-by: Runguo LI <runguo.ai@gmail.com>
* [Refactor] Reuse TransferContext in server bench Signed-off-by: riversky0014 <riversky0014@gmail.com> * Fix server bench transfer context type annotation Signed-off-by: riversky0014 <riversky0014@gmail.com> * Refine server bench block range calculation Signed-off-by: riversky0014 <riversky0014@gmail.com> * Fix engine-driven transfer dispatch by KV tensor device Signed-off-by: riversky0014 <riversky0014@gmail.com> * Reject multi-group engine-driven server bench at startup Signed-off-by: riversky0014 <riversky0014@gmail.com> --------- Signed-off-by: riversky0014 <riversky0014@gmail.com>
LocalDiskBackend, LocalCPUBackend, and NixlDynamicStorageBackend (LMCache#2966) implement batched_async_contains, so they support async loading. NixlStaticStorageBackend does not — it falls through to AbstractStorageBackend.batched_async_contains, which raises NotImplementedError, so with LMCACHE_ENABLE_ASYNC_LOADING=True and a NIXL static backend every lookup waits the full LOOKUP_TIMEOUT_MS then recomputes. Implement it with the same fail-fast batched contains() approach as the dynamic backend (LMCache#2966), bringing the static backend to parity, plus regression tests. Signed-off-by: Shuichi Ihara <sihara@ddn.com>
Signed-off-by: baoloongmao <307499405@qq.com>
…MCache#3801) Signed-off-by: nevasini1 <nevasini1@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
디자인 문서:
요약
MP mode의 L1 tier를 CPU DRAM 대신 cross-instance 공유 CXL 풀(Maru)로 교체한다.
그 위의 LMCache 동작 — L1↔L2 티어링(write-through / discard-evict / promote-on-miss),
컨트롤러, L2 어댑터, eviction — 은 기존 그대로 maru L1 위에서 돈다.
핵심 설계 결정:
컨트롤러가, MaruServer는 그 결정을 실행·기록하는 passive 공유 디렉터리(
key→region/offset+pin_count).설계 문서:
docs/design/v1/distributed/maru_l1.md(이 PR에 포함) — sibling + Protocol seam, API 계약, 상태머신 불변식, TTL sweeper·crash-recovery 전제.통합 방식 (sibling + 구조적 Protocol)
StorageManager가 config에 따라 L1을 한 줄로 선택:MaruL1Manager(cfg) if maru_config else L1Manager(cfg)— 둘 다L1ManagerInterface만족.l1_protocol.py(신규): 구조적typing.ProtocolL1ManagerInterface(17 메서드,@runtime_checkable).L1Manager·MaruL1Manager둘 다 상속 없이 구조적으로 만족.l1_manager파라미터 타입만L1ManagerInterface로확장(런타임 동작 동일).
코어 무침습
dev와 byte-identical (무변경):
l1_manager.py,l1_memory_manager.py,internal_api.py,모든 컨트롤러 로직, L2 어댑터.
실제로 편집되는 건 배선점뿐이고 전부 additive/behavior-neutral:
store/prefetch/eviction_controller.pystorage_manager.py— L1 선택 분기 + maru-onlyregister_kv_layoutwrapperconfig.py—maru_config필드 + CLI + startup guardlmcache_driven_transfer.py의register_kv_layout(default 백엔드엔 no-op)serde_wrapper.py— L2 어댑터l1_manager타입을 Protocol로 위젠(behavior-neutral)커밋 구성 (C1–C11)
L1ManagerInterfaceprotocol + maru L1 configMaruMemoryAllocator(CXL-backed allocator)MaruL1ManagerRPC control 표면 (read/write/delete/lifecycle)touch_keysreal화finish_write_and_reserve_read(L2→L1 promote, temporary/retained)l1_manager파라미터 →L1ManagerInterface위젠register_kv_layoutseamregister_kv_cache에register_kv_layout엔진 훅 배선 (+deferred 훅 테스트)테스트
maru_fakesin-memory 하네스 포함): manager 단위, Protocol 정합(test_l1_protocol),config guard, single-region, 제어 통합(기존 티어링 컨트롤러 위에서 maru 구동),
register_kv_layout,memory_allocator, conformance(같은 계약을 기존
L1Manager와MaruL1Manager에 동시 실행 → 드리프트 차단).CUDA_VISIBLE_DEVICES=1)에서 통과.범위 · 알려진 한계
init_layout에서 pool을 단일 layout으로 고정 →다른 모델·하이브리드(
num_object_groups>1)는 fail-fast ValueError. 같은 모델 멀티-인스턴스는 정상.client crash(프로세스 사망) 회수는 이 PR 범위 밖 — maru-side 전제에 의존한다:
(1) MaruServer가 client별 pin을 추적해 disconnect 시 일괄 해제, (2) 죽은 owner의 region이 RM으로 반납
(region owner-release)되며 미등록 write page 회수. 둘 다 maru-side(MaruServer) 책임.
남은 것
L1Manager에서L1StateBackendseam 추출 → 하나의L1Manager가 local/maru backend를모두 서빙(
MaruL1Manager삭제). sibling 머지 후 별도 작업.PR 초안
What this PR does / why we need it:
This PR adds Maru, a CXL shared-memory KV cache engine, as a shared L1 tier for MP mode — the shared-L1 counterpart to the in-process Maru backend from LMCache#2705. Today each MP server has a private, node-local L1 (pinned DRAM / Device-DAX / GDS). With Maru, MP servers on separate nodes
mmapone CXL pool and read each other's entries zero-copy: a node that never stored a key still hits it if a peer did, with no network transfer.Control stays entirely with LMCache. Maru is a sibling L1 manager (
MaruL1Manager) behind a new structuralL1ManagerInterfaceProtocol; the stock controllers, L1↔L2 tiering, eviction, and L2 adapters run unchanged on top. The L1 core (l1_manager.py,l1_memory_manager.py,internal_api.py) is byte-identical todev; the remaining edits are additive wiring only.Design doc:
docs/design/v1/distributed/maru_l1.md· Maru: docs / githubBenchmark:
setup
/dev/dax9.0, 500 GB), served bymaru-server+ maru-resource-managerQwen/Qwen3-8B(bf16 KV)lmcache bench engine,long-doc-qa— one exported config replayed unchanged against all three setups: 40 GB KV volume → 29 documents × 10k tokens (tokens_per_gb_kvcache=7281), 1 query/doc,tileorder, 4 in-flight, 128 output tokens (ignore_eos), seed 42--gpu-memory-utilization 0.5(GPU KV ≈ 30 GiB ≈ 220k tokens < 290k-token workload, so the GPU cache alone cannot hold the working set), default prefix caching, chunk size 256lmcache_driven): B--l1-size-gb 60(DRAM L1, default lazy allocator) · B′ same +--no-l1-use-lazy(preallocatedMixedMemoryAllocator) · C--maru-server-url maru://localhost:5555 --maru-pool-size-gb 60 --l1-size-gb 0(Maru CXL L1); same 60 GB capacity, no eviction in any runAll runs: 29/29 requests successful, 290,396 input tokens.
B′ = B with
--no-l1-use-lazy(preallocated pinned-DRAMMixedMemoryAllocatorinstead of the defaultLazyMemoryAllocator).Grafana (LMCache MP dashboard) during the runs — L1 usage filling over the warm-up pass and 100%-hit read phase:
Special notes for your reviewers:
MaruServeris only a shared directory (key → region/offset) plus a cross-nodepin_count. Page reclamation is driven by LMCache eviction, so the allocator'sfreeis a no-op.MaruMemoryAllocatorsupplies the CXL medium,MaruL1Managerthe shared control. (The MP Coordinator is unrelated: it plans fleet-wide L2 eviction on logical keys, not L1 physical-page state.) A laterL1StateBackendseam to shrink the sibling is possible follow-up, out of scope here.skip_l1store policy, registered/RDMA-type L2 (copy-type only), p2p, and engine_driven/auto transfer mode.If applicable:
docs/source/mp/configuration.rst+ design doc)