Pools: VMs, network cards and GPUs

FlashVMM pools are three local daemons that build expensive resources before they are needed and hand them out on demand: flashpoold keeps warm FlashVMs parked and ready to claim, flashnetd keeps network cards built on Linux, and flashgpud keeps models loaded on the host GPU through Metal or Vulkan, each over one strict resource-CRUD REST API (OpenAPI 3), the flash CLI and flash-mcp.

Every pool follows one model: a pool is a resource with a target, its members are resources with states, and a claim is a write to the resource being claimed. A pooled resource is exactly an ordinary one built early: a warm member is an ordinary flash run --pool FlashVM, a pooled NIC is built by the ordinary per-mode setup, and a GPU pool is served to VMs over their normal NIC. Every create through a pool still passes host admission.

Key facts

How it works

Warm VM pools (flashpoold)

A pool has one caller-chosen shape (vcpus, mem_mib, optional disk, CPU, NIC and GPU caps) and a target_count of warm members. Each member boots as flash run --pool: it pays kernel and guest-agent start-up, reaches the agent's Ready and parks with zero CPU. A claim is PUT /v1/flashvm/workload on that member's own socket: it hands the spare its entry, re-targets its disk and CPU caps to the claimed workload, and wakes it. The pool API never hands out workloads; it only builds, lists and tears down members.

A claim takes the member out of the warm set and the daemon boots a replacement. A finished claimed member is removed, and a claimed member is never deleted to shrink the warm set. A member that never parks, dies booting or dies while parked backs the pool off exponentially; after 10 failures in a row it stops refilling (refill_halted, with its reason, persisted across restarts) until PATCH {"refill_halted": false}. No member launches unless host available memory covers its mem_mib plus the daemon's --min-free-mib floor. Each OS uses its native path: on Linux a member's NAT NIC comes from flashnet and it is idle once the guest reports a metrics sample; on macOS (HVF) the daemon runs per user, the NIC is built-in user-mode NAT, and a member is idle once it reports parked; on Windows (WHP) the daemon serves one named pipe, each member its own \\.\pipe\flashvm-<id>, with built-in user-mode NAT.

NIC pools (flashnetd, Linux)

flashnetd is one root daemon on /run/flashnetd.sock that builds count NICs of one mode (nat, routed or bridge) before any VM starts, each tap owned by the pool's uid, with an optional egress policy every claimant gets. flash run --nic pool=ID claims one (POST /v1/nic-pools/{id}/claims) and receives the ready --nic value. The NIC returns to the pool when its VMM lets go of the tap, or on DELETE of the claim, and is rebuilt clean: NAT and routed NICs get a new address and fresh firewall state, bridge NICs cycle the link to flush neighbor and forwarding entries. State lives on disk, so a restarted daemon re-adopts its pools. See networking.

GPU pools (flashgpud)

A GPU pool is one model (a GGUF file on the host) loaded by an engine on the host GPU and shared by any number of VMs. No VM gets a GPU device: a VM created with --gpu-pool receives a grant, and its workload gets FLASH_GPU_URL, the pool's chat-completions HTTP endpoint reached over its NIC, and FLASH_GPU_TOKEN, its bearer token. The guest endpoint checks the request head and token, then streams bytes both ways untouched, so large bodies and streamed (SSE) replies pass straight through. The engine serves a fixed number of parallel slots; beyond that, the next free slot goes round-robin to the next grant with a request waiting, so one busy VM cannot starve the others. Revoking a grant stops its token at once. Because the GPU stays on the host, snapshots, forks and warm pools of GPU-using VMs work unchanged.

Interfaces

CLI

flash pool create --count 8 --vcpus 1 --mem-mib 256 --nic kind=nat --wait-idle 60
flash pool members web --state idle
flash claim web-3 --entry "python3 /work/job.py"
flash pool set web --target 16
flash pool set web --resume-refill
sudo flashnet pool create --mode nat --count 32
flash run --bundle ./bundle --vcpus 1 --mem-mib 256 --nic pool=ab12cd
flash gpu pool create --model /models/model.gguf --slots 4 --wait-ready 120
flash run --bundle ./bundle --vcpus 2 --mem-mib 1024 --nic tap=tap0,ip=10.0.0.2/30 --gpu-pool llm
flash gpu grant rm job1

MCP

flash_pool(method: "POST", path: "/v1/vm-pools?wait=idle&timeout_s=60", body: {"shape": {"vcpus": 1, "mem_mib": 256, "nics": [{"kind": "nat"}]}, "count": 8})
flash_api(vm_id: "web-3", method: "PUT", path: "/v1/flashvm/workload", body: {"entry": "python3 /work/job.py"})
flash_gpu(method: "POST", path: "/v1/gpu-pools?wait=ready&timeout_s=120", body: {"model": "/models/model.gguf", "slots": 4})

REST

POST   /v1/vm-pools {"shape": {"vcpus": 1, "mem_mib": 256}, "count": 8}
PATCH  /v1/vm-pools/{pool_id} {"target_count": 16}
GET    /v1/vm-pools/{pool_id}/members?state=idle
PUT    /v1/flashvm/workload {"entry": "python3 /work/job.py"}        (on the member's socket)
POST   /v1/nic-pools {"mode": "nat", "count": 32}
POST   /v1/nic-pools/{pool_id}/claims {"vm_id": "job1"}
POST   /v1/gpu-pools {"model": "/models/model.gguf", "slots": 4}
POST   /v1/gpu-grants {"vm_id": "job1", "gpu_pool": "llm"}
DELETE /v1/gpu-grants/{vm_id}

Contract an agent must know: a claim on a VM not booted with --pool returns 409; adding members to a draining or deleting pool returns 409 pool_draining or pool_deleting; a bad shape returns 422 invalid_shape; a pool write without operator rights returns 403 forbidden; a NIC claim with no free NIC returns 409; POST /v1/gpu-grants is the only reply that carries the bearer token, and returns 409 while that VM still holds a grant; every daemon returns 503 busy when too many requests are in flight.

Guarantees

• Pooled is ordinary. A warm VM, a pooled NIC and a GPU-backed VM behave exactly like their unpooled forms.
• Claims are writes to the claimed resource. One PUT on the member's socket assigns its workload.
• Pools heal and protect the host. Replacements boot on every claim; failures back off and halt; a memory floor gates every launch.
• NICs come back clean. Every returned NIC is rebuilt with fresh address, firewall and forwarding state.
• Fair GPU sharing. Round-robin slots per grant, revocable tokens, and VMs that stay snapshot-ready.