Memory model and dirty tracking

The FlashVMM memory model is how a FlashVM's guest RAM is laid out, backed, tracked and reclaimed: memory mapped on demand and never committed up front, dirty pages tracked at 4 KiB granularity including the VMM's own writes, and pages returned to the host live through virtio-balloon and virtio-mem.

Guest RAM costs the host only the pages the guest touches, so an idle VM given 256 MB uses as little as 27 MB of real host memory. The same page-level bookkeeping that keeps memory cheap is what makes diff snapshots, checkpoints and live fork store only what changed. See snapshots and fork and checkpoints.

Key facts

How it works

Backing

On Linux every guest RAM region is an mmap with MAP_NORESERVE, so nothing is committed or prefaulted at create; a page exists on the host once the guest first touches it. Three backings share one layout function: anonymous memory for a cold boot, a MAP_SHARED RAM file for a --forkable VM, and a MAP_PRIVATE copy-on-write mapping of a snapshot for a clone, so many VMs restored from one file share its physical pages and diverge only where they write. A demand-paged restore fills RAM through userfaultfd instead (see snapshots).

Layout

The snapshot RAM section is always contiguous from 0 to the memory size; the layout maps it to guest addresses. On x86_64 a guest above 3328 MiB is split into a region below the 32-bit device window and one starting at 4 GiB; on arm64 RAM is one region at the platform's RAM base. KVM slots, the e820 map, snapshot capture and restore, copy-on-write mapping and the userfaultfd fault handler all use the same layout.

Dirty tracking

Each KVM memory slot is created with KVM_MEM_LOG_DIRTY_PAGES, so the kernel records every page the guest writes. Writes the VMM makes into guest RAM itself (virtqueue used rings, received network frames, block and 9p read completions, rng fills, balloon zeroing) are recorded in a shared host-dirty set that each device marks as it writes. A diff capture reads both and folds them together. The macOS hvf engine tracks dirty pages for its own snapshots and forks.

Reclaim

On Linux and macOS every VM has a virtio-balloon: flash balloon --target-pages N asks the guest to hand pages back, and the host discards them with MADV_DONTNEED, or with MADV_REMOVE while RAM is a shared file, so the memory is actually freed. The reply reports what was asked beside what was returned. A VM created with --mem-region-mib also has a virtio-mem region whose live target is set with flash mem --target-mib; the guest plugs or unplugs whole blocks. See limits and memory features.

Interfaces

CLI

flash run --bundle ./mem-enabled-bundle --id db --vcpus 2 --mem-mib 1024 --mem-region-mib 2048 --entry 'sleep 3600' --detach
flash balloon db --target-pages 16384
flash balloon db
flash mem db --target-mib 1024
flash mem db

MCP

flash_resource(vm_id: "db", resource: "memory")
flash_api(vm_id: "db", method: "PATCH", path: "/v1/flashvm/memory/balloon", body: {"target_pages": 16384})
flash_resource(vm_id: "db", resource: "balloon")
flash_api(vm_id: "db", method: "PATCH", path: "/v1/flashvm/memory/mem", body: {"requested_mib": 1024})
flash_resource(vm_id: "db", resource: "mem")

REST

GET   /v1/flashvm/memory
PATCH /v1/flashvm/memory/balloon  {"target_pages": 16384}
GET   /v1/flashvm/memory/balloon    → {"target_pages": 16384, "freed_pages": ...}
PATCH /v1/flashvm/memory/mem      {"requested_mib": 1024}
GET   /v1/flashvm/memory/mem        → {"region_mib", "usable_region_mib", "block_kib", "requested_mib", "plugged_mib"}

Contract: memory/mem returns 422 mem_not_attached on a VM created without a region and 422 mem_exceeds_region for a target past usable_region_mib; a balloon change on a paused VM returns 409 vm_paused. A VM created with --disable-dirty-tracking takes full snapshots.

Guarantees

• Pay for what is touched. Guest RAM is reserved as address space and backed page by page on first use.
• Complete diffs. Guest writes and the VMM's own writes are both tracked, so a diff holds every changed page.
• Reclaim that frees memory. Ballooned pages are discarded with the advice that releases them for the current backing.
• Reported, not assumed. Balloon and virtio-mem replies show the request beside what the guest actually did.