Skip to content

fix(graph): deduplicate shared mmap weight files across component graphs - #794

Open
Moeez34 wants to merge 1 commit into
cactus-compute:mainfrom
Moeez34:main
Open

fix(graph): deduplicate shared mmap weight files across component graphs#794
Moeez34 wants to merge 1 commit into
cactus-compute:mainfrom
Moeez34:main

Conversation

@Moeez34

@Moeez34 Moeez34 commented Aug 20, 2026

Copy link
Copy Markdown

Problem

When loading models composed of multiple component graphs (e.g. lm_encoder, lm_encoder_text_chunk, lm_encoder_step for models like gemma-4-e2b-it), shared weight files like embed_tokens_per_layer.weights are bound by multiple components.

Previously, weight_cache_ was isolated to each individual CactusGraph instance. As a result, each component graph created its own MappedFile object and issued an independent mmap() system call for shared weights.

On operating systems with per-process virtual address space / mapped-memory ceilings (such as iOS's ~6.5 GiB limit), repeatedly mapping a 1.13 GiB weight file 3+ times alongside other weights exhausts the process address space budget. This caused mmap to fail with ENOMEM ("Cannot map file") during cactus_init.

Solution

  1. Process-Wide Thread-Safe Mmap Cache (MappedFileRegistry):

    • Introduced GraphFile::MappedFileRegistry in cactus_graph.h / io.cpp.
    • Maintains a thread-safe registry mapping canonical absolute file paths (std::filesystem::canonical) to std::weak_ptr<GraphFile::MappedFile>.
    • If a weight file is already memory-mapped by any component graph in the process, subsequent requests share the active MappedFile instance instead of creating duplicate virtual memory mappings.
  2. Refactored CactusGraph Storage:

    • Updated CactusGraph::mapped_files_ from std::vector<std::unique_ptr<MappedFile>> to std::vector<std::shared_ptr<MappedFile>>.
    • Updated mmap_embeddings, mmap_weights, bind_mmap_weights, and embedding to retrieve shared MappedFile pointers via MappedFileRegistry::get_or_load().
  3. Automatic Cleanup (RAII):

    • When all component graphs referencing a weight file are destroyed or reset, the std::shared_ptr reference count reaches zero, triggering MappedFile::~MappedFile() (munmap), cleanly unmapping the region.

Verification

  • Added test_shared_mmap_weights_dedup() unit test in cactus-graph/tests/test_io.cpp.
  • Verified that multiple CactusGraph instances binding the same weight file receive identical mmap buffer pointers (ptr1 == ptr2) and release the mapping safely upon graph teardown.

Signed-off-by: Moeez Ahmed <ahmedmoeez7860@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant