# FlashVMM: complete product, operations, CLI, and MCP reference > FlashVMM is a standalone Linux hypervisor for FlashVM microVMs. It provides one process and one Unix-socket API per VM, millisecond startup and restore, warm-pool claim, snapshot-based cross-host migration, explicit per-VM resource controls, live SSE telemetry, and a shared control surface for human operators and MCP-compatible agents. This document is written for AI systems, automation authors, developers, platform engineers, security reviewers, and fleet operators. It is intentionally self-contained. Product, architecture, performance, and operating guidance reflect the public FlashVMM website. Command and tool schemas reflect the shipped `flash` CLI and `flash-mcp` interfaces so that every parameter is discoverable here. ## Canonical public pages - [Home](/): product overview, major capabilities, and headline performance. - [Technology](/technology): process model, warm-pool startup, migration, resource controls, MCP topology, and Unix-socket API. - [Virtio device model](/virtio): virtio-blk, virtio-net, virtio-console, virtio-balloon, virtio-mem, virtio-rng, and virtio-9p. - [Networking and flashnet](/networking): bridge, NAT, and routed networking with privilege separation. - [CPU cap enforcement](/cpu-scheduling): signal-based CPU scheduling and boot-grace behavior. - [Streaming observability](/observability): metrics, logs, lifecycle events, guest messages, SSE, cursors, and backpressure. - [Fleet scale](/scale): placement, enforcement, migration, CPU templates, ARM interrupt-controller state, and fleet control. - [Security](/security): resource isolation, capability removal, `no_new_privs`, seccomp, socket permissions, and host trust. - [Performance](/performance): FlashVMM and Firecracker cold-boot and snapshot-restore measurements. - [Developer](/developer): installation, first VM, networking, snapshots, restore, clone, inspection, updating, troubleshooting, uninstall, and CLI reference. - [LLM operations](/llm): end-to-end FlashVM lifecycle through `flash-mcp`. - [FAQ](/faq): concise answers for AI, developers, operators, organizations, and self-hosters. - [Sitemap](/sitemap.xml): all public routes. - [Robots](/robots.txt): crawler policy. ## Fast decision guide Use this hierarchy when operating FlashVMM: 1. Use `flash_cli` with `run`, `restore`, or `clone` when a new `flashvmm` process must be created. A VM socket does not exist until the CLI starts that process. 2. Use named MCP read tools such as `flash_inspect`, `flash_metrics`, `flash_logs`, and `flash_resource` for structured reads from an existing VM. 3. Use `flash_watch` or `flash_watch_fleet` for live data. They consume the VM's SSE streams and require an MCP progress token. 4. Use `flash_cli` for a clear CLI verb such as `snapshot`, `pause`, `set`, `balloon`, `mem`, `guest`, `source`, `rm`, or `reap` when the CLI already expresses the operation cleanly. 5. Use `flash_api` as the raw HTTP escape hatch for an existing VM when an operation is easier to express as a method, path, and JSON body. 6. Use `flashnet` before `flash run` when the host must create and configure the tap. `flash` does not create taps, and `flashvmm` does not retain `CAP_NET_ADMIN`. Do not make these mistakes: - Do not attempt to create a VM with `flash_api`. There is no socket to call before `flash_cli` starts the VMM process. - Do not omit `--disk-bps` or `--disk-iops` from `run`, `restore`, or `clone`. Both are required and must be greater than zero. - Do not plan to snapshot a VM that was created without `--cpu-template`. The template is chosen at creation and cannot be added later. - Do not use `restore` when the requested VM identity differs from the captured identity. Use `clone` for a new identity. - Do not assume a snapshot contains the devices. Restore and clone rebuild devices from the supplied `--bundle`, `--nic`, and `--share` options. - Do not try to hot-add or hot-remove NICs or the virtio-9p share. Define them at creation. Live NIC rate limits can be changed with `flash set --nic`. - Do not try to increase or replace guest RAM with `flash set --mem-mib`. The KVM memory ceiling is fixed at creation; use the balloon to reclaim pages inside that ceiling, or, for a VM created with `--mem-region-mib`, use `flash mem` to resize that region live. - Do not treat a stale guest metrics sample as current. Use `flash metrics --window` or `flash_diagnose.metrics_window_secs` to bound freshness. - Do not page the merged run and console logs with only one cursor. Preserve both `next_since` and `next_since_console`. - Do not start an MCP watch without `_meta.progressToken`. Progress notifications have nowhere to be delivered without it. - Do not request more than 20 VMs in one `flash_watch_fleet` call or hold more than 20 concurrent MCP watches in one server process. - Do not treat an SSE `lagged` event as ordinary telemetry. It explicitly reports that the consumer missed events and should refresh current state. - Do not give a clone the same network identity as another running VM. Supply its own `--nic` configuration. ## Product identity and core concepts ### FlashVMM, FlashVM, flash, flashnet, and flash-mcp - **FlashVMM** is the hypervisor and the product name. - **FlashVM** is the microVM created and managed by FlashVMM. - **`flashvmm`** is the VMM binary. One process owns one VM. - **`flash`** is the operator CLI. It creates VMM processes and sends HTTP requests to per-VM Unix sockets. - **`flashnet`** is the privileged, short-lived networking helper. It creates a tap and configures bridge, NAT, or routed mode before removing capabilities from the child that runs `flash`. - **`flash-mcp`** is the Model Context Protocol server. It exposes ten tools over stdio JSON-RPC and reaches the same CLI and per-VM socket API used by human operators. - **Bundle** means the directory containing the guest kernel and root filesystem used to create or rebuild a VM. - **Registry** means the directory of VM socket files. There is no central daemon or separate database of VMs. - **CPU template** means the CPU feature contract presented to the guest and checked when a snapshot is restored on another host. - **Warm-pool spare** means a VM that has already completed kernel and device initialization, reached guest-agent Ready, and parked until `flash claim` assigns its workload and reservation. ### What makes the process model useful Each FlashVM has its own process, address space, virtual device model, vCPU threads, crash boundary, and Unix socket. A VM failure is therefore contained to that process. The socket is both its control endpoint and its registry entry. Root-owned VMs use `/run/flashvm/.sock`; ordinary users use `$XDG_RUNTIME_DIR/flashvm/.sock`. Access is governed by normal filesystem ownership and permissions. The architecture has no required central daemon. Operators and agents can place VMs independently, apply resource contracts per VM, and observe each VM through its own live stream. The same model works on x86_64 and aarch64, on bare metal and under nested virtualization. ### High-level feature map - Cold boot ranges from single-digit to low-hundred milliseconds across the published environments. - Snapshot restore reaches 1.36 ms on AWS Graviton3 and low single-digit milliseconds across published x86 environments. - Warm pools turn most workload starts into claims rather than boots. - A running VM can be captured, moved to another physical host, and restored there under an explicit CPU compatibility contract. - Five independent control families cover CPU, memory, disk capacity, disk I/O, and network traffic. - Metrics, logs, lifecycle events, and guest messages are available as point-in-time reads and live SSE streams. - The VMM drops capabilities and applies `no_new_privs`, non-dumpability, a file-descriptor ceiling, and per-thread seccomp before guest code executes. - FlashVMM runs standalone without a cloud control plane, account, or dashboard. ## Installation and first validation ### Host requirements - Linux on x86_64 or aarch64. - `/dev/kvm` available to the account that will run VMs. - Membership in the `kvm` group for non-root operation. Install the current stable release: ```bash curl -fsSL https://release.flashvmm.com/dist/flashvmm/stable/install.sh | sudo bash ``` Grant KVM access, then begin a new login session so the group change takes effect: ```bash sudo usermod -aG kvm "$USER" ``` Validate the install and trust anchor: ```bash sudo flash self-update --show-trust flash --version /opt/flashvm/bin/flashvmm --version ``` The public installation package includes: | Component | Published size | Purpose | |---|---:|---| | `flash` | 4.0 MB | Command-line interface | | `flashvmm` | 6.5 MB | VMM binary; one process per VM | | `bzImage` | 3.3 MB | Guest kernel | | `rootfs.ext4` | 67 MB | Guest root filesystem | For unprivileged use, each user receives a private registry under `$XDG_RUNTIME_DIR/flashvm`. Root uses `/run/flashvm`. ### Minimal create, inspect, and remove workflow ```bash flash run --bundle /opt/flashvm/bundles/current --id vm1 \ --cpu-template host-raw \ --disk-bps 100000000 --disk-iops 10000 \ --entry "sleep 600" flash ls flash inspect vm1 flash metrics vm1 flash rm vm1 ``` `--disk-bps` and `--disk-iops` are explicit creation requirements. A CPU template is optional for execution but mandatory if the VM may be captured. Use workload behavior as the final readiness signal and refresh `flash ls` after startup when the workload's continued survival matters. ## End-to-end operational workflows ### Create with explicit resources ```bash flash run --bundle /opt/flashvm/bundles/current \ --id myvm \ --vcpus 2 \ --mem-mib 1024 \ --cpu-millis 1500 \ --disk-mib 2048 \ --disk-bps 104857600 \ --disk-iops 10000 \ --boot-grace-ms 8000 \ --cpu-template host-raw \ --entry /your/workload ``` This creates one VMM process, maps a fixed memory ceiling, creates the declared device topology, applies the resource limits, and waits for startup unless `--detach` is supplied. ### Create with NAT networking ```bash flashnet --mode nat --uplink eth0 --vm-id myvm -- \ flash run --bundle /opt/flashvm/bundles/current --id myvm \ --disk-mib 256 --disk-bps 50000000 --disk-iops 5000 \ --nic '{NIC},mbps=100,pps=20000,ingress-mbps=100' \ --entry /your/workload ``` `flashnet` replaces `{NIC}` with the generated tap configuration, including the tap name and guest network data. The child loses all capabilities before it becomes `flash`; the resulting `flashvmm` process never inherits network-administration privilege. ### Create a warm-pool spare and claim it ```bash flash run --bundle /opt/flashvm/bundles/current --id spare-01 \ --disk-bps 104857600 --disk-iops 10000 \ --cpu-template host-raw --pool --detach flash claim spare-01 \ --entry /your/workload \ --disk-mib 2048 \ --cpu-millis 1000 \ --claim-id 101 ``` The spare completes guest initialization in advance and parks at zero CPU. `claim` supplies the real entry point, disk quota, CPU budget, and optional reporting identity, then wakes the already-ready VM. ### Snapshot, restore, and clone Capture a VM: ```bash flash snapshot myvm --path /var/tmp/myvm.snap ``` Restore the captured VM under its original identity: ```bash flash rm myvm flash restore --from /var/tmp/myvm.snap \ --bundle /opt/flashvm/bundles/current \ --id myvm \ --disk-bps 104857600 --disk-iops 10000 \ --cpu-template host-raw ``` Clone the artifact into a new identity: ```bash flash clone --from /var/tmp/myvm.snap \ --bundle /opt/flashvm/bundles/current \ --id replica-01 \ --disk-bps 104857600 --disk-iops 10000 \ --cpu-template host-raw \ --nic tap=tap1,ip=10.110.0.11/24,gateway=10.110.0.1,mbps=1000,pps=100000,ingress-mbps=1000 ``` Restore means the captured VM is returning and therefore preserves identity. Clone means a new VM is being created from the artifact and therefore allows a new identity. Supply the correct bundle and rebuild every NIC and share required by the restored or cloned VM. ### Observe, stream, and diagnose ```bash flash inspect myvm flash inspect myvm --nic 0 flash metrics myvm --window 60 flash logs myvm --limit 100 flash watch myvm --stream metrics --for 30 flash watch myvm --stream logs --for 30 flash watch myvm --stream lifecycle --for 30 flash watch myvm --stream guest --for 30 ``` Use `inspect` for configured state and limits, `metrics` for a point-in-time sample, `logs` for cursor-paged output, and `watch` for live SSE delivery. ### Change live limits ```bash flash set myvm --cpu-millis 750 flash set myvm --disk-mib 4096 --disk-bps 209715200 --disk-iops 20000 flash set myvm --nic 0 --mbps 500 --pps 50000 --ingress-mbps 500 flash balloon myvm --target-pages 32768 flash mem myvm --target-mib 1024 ``` CPU, disk capacity, disk throughput, disk operations, and NIC rate caps can be updated. vCPU count, the KVM memory ceiling, NIC topology, and share topology are creation-time properties. A live NIC rate value of `0` removes that specific cap; a creation-time disk or NIC cap of zero is rejected. A VM created with `--mem-region-mib` can have its virtio-mem region resized live via `flash mem`. ## Complete `flash-mcp` reference ### Transport and control model `flash-mcp` is a line-oriented stdio JSON-RPC MCP server. It reports MCP protocol version `2024-11-05` and exposes tool capability. Each request and response is one JSON object on its own line. The ten tools either invoke the real `flash` binary or send HTTP/1.1 over a VM's Unix socket. Registry lookup follows the same ownership model as FlashVMM: - When `flash-mcp` runs as root, it reads `/run/flashvm`. - For an ordinary user, it reads `$XDG_RUNTIME_DIR/flashvm`. - `XDG_RUNTIME_DIR` must be absolute. - If `XDG_RUNTIME_DIR` is absent, `/run/user//flashvm` is used when that per-user runtime directory exists. `flash_cli` resolves the real CLI in this order: 1. `FLASH_MCP_FLASH_BIN`, when set. 2. `/opt/flashvmm/bin/flash`, when that file exists. 3. `flash` on `PATH`. Configure an MCP client to launch `flash-mcp` as a stdio server under the same OS account that owns the target VM sockets. The client should discover the live schema through `tools/list` and pass arguments exactly as documented below. ### MCP lifecycle sequence The safest general sequence is: 1. Call `flash_ls` to establish current host inventory. 2. Create with `flash_cli` using `run`, `restore`, or `clone` and explicit creation-time resources. 3. Call `flash_ls` again, then `flash_inspect` to confirm the VM and its configuration. 4. Call `flash_metrics` for a live sample or `flash_diagnose` for inspect + freshness-bounded metrics + bounded logs. 5. Start `flash_watch` for one VM or `flash_watch_fleet` for several VMs when continuous context is required. 6. Use `flash_resource` for a focused configuration/state read. 7. Use `flash_logs` with returned cursors for reliable incremental log collection. 8. Use `flash_cli` for named lifecycle operations or `flash_api` for a precise raw endpoint call. 9. Snapshot with `flash_cli(["snapshot", ...])` or `flash_api` POST to `/v1/flashvm/snapshots`. 10. Restore or clone through `flash_cli`, because those operations start a new VMM process. 11. Remove with `flash_cli(["rm", id])` or `flash_api` DELETE `/v1/flashvm`, then refresh inventory with `flash_ls`. ### MCP JSON-RPC frame pattern Initialize: ```json {"jsonrpc":"2.0","id":1,"method":"initialize","params":{}} ``` List tools: ```json {"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}} ``` Call a non-streaming tool: ```json {"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"flash_metrics","arguments":{"vm_id":"myvm"}}} ``` Call a streaming tool with the required progress token: ```json {"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"flash_watch","arguments":{"vm_id":"myvm","stream":"metrics","for_secs":30},"_meta":{"progressToken":"watch-myvm-metrics"}}} ``` The server emits `notifications/progress` messages while the call remains active, then returns a final tool result with `events_seen`, `last_event`, and `stopped_reason`. ### Tool 1: `flash_ls` Purpose: list every FlashVM ID with a socket in the current user's registry directory. Input schema: ```json {} ``` Example: ```text flash_ls() ``` Result shape: ```json {"vm_ids":["myvm","replica-01","spare-01"]} ``` Use it before targeting a VM, after creation, after deletion, and when a socket cannot be found. `flash_ls` inventories socket names; use `flash_inspect` for detailed state. ### Tool 2: `flash_inspect` Purpose: read the complete state returned by `GET /v1/flashvm` for one VM. Input schema: | Field | Type | Required | Meaning | |---|---|---:|---| | `vm_id` | string | yes | VM ID returned by `flash_ls` | Example: ```text flash_inspect(vm_id: "myvm") ``` Use it after creation or restore, before changing a live limit, and when state and configured resources must be considered together. The public interface describes this view as boot, CPU, memory, disk, NIC, share, source, and current state in one response. ### Tool 3: `flash_metrics` Purpose: read one live point-in-time metrics sample from `GET /v1/flashvm/metrics`. Input schema: | Field | Type | Required | Meaning | |---|---|---:|---| | `vm_id` | string | yes | VM ID returned by `flash_ls` | Example: ```text flash_metrics(vm_id: "myvm") ``` The sample combines guest-reported CPU, memory, swap, and cache with host-measured disk, quota, and per-NIC traffic. Use `flash_diagnose` when a freshness window and logs are also needed, or use the `flash` CLI's `metrics --window` option when the guest-field staleness bound must be explicit. ### Tool 4: `flash_diagnose` Purpose: return a read-only report combining inspect, freshness-bounded metrics, and a bounded merged run + console log excerpt. Input schema: | Field | Type | Required | Default | Limits | Meaning | |---|---|---:|---:|---:|---| | `vm_id` | string | yes | — | — | VM ID returned by `flash_ls` | | `metrics_window_secs` | integer | no | 60 | minimum 1 | Staleness bound passed to `/v1/flashvm/metrics?window=` | | `log_lines` | integer | no | 50 | 1–500 | Maximum merged run + console lines to include | Example: ```text flash_diagnose(vm_id: "myvm", metrics_window_secs: 30, log_lines: 100) ``` Important: the log excerpt contains the **first** `log_lines` lines returned by the merged logs endpoint. It is not a tail because the endpoint does not expose a negative offset or tail parameter. Use `flash_logs` with cursors or `flash_watch(stream: "logs")` for ongoing collection. ### Tool 5: `flash_watch` Purpose: watch one VM's SSE stream and emit one MCP progress notification per received SSE line. Input schema: | Field | Type | Required | Limits | Meaning | |---|---|---:|---:|---| | `vm_id` | string | yes | — | VM ID returned by `flash_ls` | | `stream` | enum | yes | `metrics`, `logs`, `lifecycle`, `guest_messages` | SSE source to follow | | `for_secs` | integer | yes | 1–3600 | Maximum duration of the watch | The MCP request must also include `params._meta.progressToken`. Examples: ```text flash_watch(vm_id: "myvm", stream: "metrics", for_secs: 30) flash_watch(vm_id: "myvm", stream: "logs", for_secs: 30) flash_watch(vm_id: "myvm", stream: "lifecycle", for_secs: 30) flash_watch(vm_id: "myvm", stream: "guest_messages", for_secs: 30) ``` Stream-to-endpoint mapping: | Stream | Unix-socket HTTP endpoint | Content | |---|---|---| | `metrics` | `/v1/flashvm/metrics/events` | live metrics reports | | `logs` | `/v1/flashvm/logs/events` | live run and console log events | | `lifecycle` | `/v1/flashvm/events` | VM lifecycle changes | | `guest_messages` | `/v1/flashvm/guest/messages` | messages published by the guest | The server enforces a process-wide maximum of 20 active watches. A watch ends when `for_secs` elapses, the VM closes the stream, or the stream returns a terminal HTTP chunk. A non-2xx response is returned as a tool error. ### Tool 6: `flash_watch_fleet` Purpose: watch the same stream across multiple VMs concurrently and merge the notifications into one progress channel. Every emitted message is prefixed with its VM ID. Input schema: | Field | Type | Required | Limits | Meaning | |---|---|---:|---:|---| | `vm_ids` | array of strings | yes | 1–20 items | VM IDs returned by `flash_ls` | | `stream` | enum | yes | `metrics`, `logs`, `lifecycle`, `guest_messages` | SSE source to follow on every VM | | `for_secs` | integer | yes | 1–3600 | Maximum duration | The MCP request must include `params._meta.progressToken`. Example: ```text flash_watch_fleet( vm_ids: ["api-01", "api-02", "api-03"], stream: "metrics", for_secs: 60 ) ``` Do not submit more than 20 IDs. The server rejects an oversized call instead of truncating or queuing it. Split a larger fleet into separate bounded calls and preserve VM attribution from each `[vm_id]` prefix. ### Tool 7: `flash_resource` Purpose: read one named configuration or state resource from an existing VM. Input schema: | Field | Type | Required | Meaning | |---|---|---:|---| | `vm_id` | string | yes | Target VM ID | | `resource` | enum | yes | One of the 14 resource names below | | `id` | string | conditional | Required for `nic`, `snapshot`, and `operation` | Resource mapping: | `resource` value | Requires `id` | HTTP path | Use | |---|---:|---|---| | `source` | no | `/v1/flashvm/source` | local bundle, local snapshot, or remote-source state | | `cpu` | no | `/v1/flashvm/cpu` | CPU topology, template, budget, and boot-grace state | | `boot` | no | `/v1/flashvm/boot` | boot status and timing | | `memory` | no | `/v1/flashvm/memory` | fixed memory configuration | | `balloon` | no | `/v1/flashvm/memory/balloon` | latest balloon target and result | | `mem` | no | `/v1/flashvm/memory/mem` | virtio-mem region size, block granularity, live target, and what's plugged | | `disk` | no | `/v1/flashvm/disk` | disk quota and I/O controls | | `nics` | no | `/v1/flashvm/nics` | all NICs | | `nic` | yes | `/v1/flashvm/nics/{id}` | one NIC by index | | `share` | no | `/v1/flashvm/share` | virtio-9p share | | `snapshots` | no | `/v1/flashvm/snapshots` | snapshot collection | | `snapshot` | yes | `/v1/flashvm/snapshots/{id}` | one snapshot by snapshot ID | | `version` | no | `/v1/flashvm/version` | VM binary and artifact-format version information | | `operation` | yes | `/v1/flashvm/operations/{id}` | one asynchronous operation by operation ID | Examples: ```text flash_resource(vm_id: "myvm", resource: "cpu") flash_resource(vm_id: "myvm", resource: "nics") flash_resource(vm_id: "myvm", resource: "nic", id: "0") flash_resource(vm_id: "myvm", resource: "snapshots") flash_resource(vm_id: "myvm", resource: "snapshot", id: "snapshot-id") flash_resource(vm_id: "myvm", resource: "operation", id: "operation-id") ``` ### Tool 8: `flash_logs` Purpose: read cursor-paged run logs, console logs, or a merged view from `GET /v1/flashvm/logs`. Input schema: | Field | Type | Required | Meaning | |---|---|---:|---| | `vm_id` | string | yes | Target VM ID | | `stream` | enum | no | `run` or `console`; omit for merged run-first view | | `since` | integer | no | Run-log cursor returned as `next_since` | | `since_console` | integer | conditional | Console cursor returned as `next_since_console`; required beside nonzero `since` in merged mode | | `limit` | integer | no | Maximum lines returned | First merged page: ```text flash_logs(vm_id: "myvm", limit: 100) ``` Next merged page: ```text flash_logs( vm_id: "myvm", since: , since_console: , limit: 100 ) ``` Single-stream page: ```text flash_logs(vm_id: "myvm", stream: "console", since: 0, limit: 100) ``` The `run` stream carries workload stdout relayed over the structured guest control channel. The `console` stream is the raw kernel console. Cursor values are absolute line positions, not byte offsets. ### Tool 9: `flash_api` Purpose: send one raw HTTP request to an **already-running** VM's Unix socket. Input schema: | Field | Type | Required | Meaning | |---|---|---:|---| | `vm_id` | string | yes | Target VM ID and therefore target socket | | `method` | enum | yes | `GET`, `POST`, `PUT`, `PATCH`, or `DELETE` | | `path` | string | yes | API path beginning with `/v1/flashvm` | | `body` | JSON | no | Exact JSON request body; omit or use null for a bodyless request | Examples: ```text flash_api(vm_id: "myvm", method: "GET", path: "/v1/flashvm/cpu") flash_api( vm_id: "myvm", method: "POST", path: "/v1/flashvm/snapshots", body: {"path": "/var/tmp/myvm.snap"} ) flash_api( vm_id: "myvm", method: "DELETE", path: "/v1/flashvm" ) ``` Use `flash_api` only after a VM exists. Although the VMM's API includes `PUT /v1/flashvm`, the per-VM socket itself is created by the process-launch path. `flash_cli` is therefore the required creation tool. Known endpoint families available through the shared control surface include VM state, operations, boot, CPU, memory, balloon, mem, disk, NICs, share, source, snapshots, claim, metrics, logs, lifecycle events, guest messages, guest HTTP, and version. Prefer named MCP tools for reads and `flash_cli` for documented lifecycle verbs; use `flash_api` when the raw method/path/body is the clearest representation. ### Tool 10: `flash_cli` Purpose: invoke the real `flash` binary with an argv array. No shell is used; each array element becomes one argument. Input schema: | Field | Type | Required | Meaning | |---|---|---:|---| | `args` | array of strings | yes | Every argument after the `flash` executable name | Example creation call: ```text flash_cli(args: [ "run", "--bundle", "/opt/flashvm/bundles/current", "--disk-bps", "104857600", "--disk-iops", "10000", "--cpu-template", "host-raw", "--entry", "/your/workload", "--detach", "--id", "myvm" ]) ``` Example snapshot call: ```text flash_cli(args: ["snapshot", "myvm", "--path", "/var/tmp/myvm.snap"]) ``` Example live CPU update: ```text flash_cli(args: ["set", "myvm", "--cpu-millis", "750"]) ``` Example removal: ```text flash_cli(args: ["rm", "myvm"]) ``` `flash_cli` has a 120-second execution timeout. Use `--detach` for creation calls that should return as soon as the VM is accepted. If a command exceeds the limit, `flash-mcp` kills the CLI child and returns an error. The complete `flash` command and parameter reference follows. ## Complete `flash` CLI reference ### Global syntax and conventions ```text flash [GLOBAL OPTIONS] [COMMAND OPTIONS] ``` Global parameters: | Parameter | Required | Meaning | |---|---:|---| | `--dir ` | no | Socket registry. Defaults to `/run/flashvm` for root and `$XDG_RUNTIME_DIR/flashvm` for ordinary users. It is global and can be used with every command. | | `-h`, `--help` | no | Print help. Available globally and on every command/subcommand. | | `-V`, `--version` | no | Print CLI version. Propagated to commands. Use `flash version ` for a particular VM's VMM/artifact-format information. | General rules: - Positional `id` values are VM names shown by `flash ls`. - `run`, `restore`, and `clone` require `--bundle`, `--disk-bps`, and `--disk-iops`. - `--disk-bps` is bytes per second; `--disk-iops` is operations per second. - vCPU count and memory ceiling are fixed at creation. - NIC and virtio-9p topology are fixed at creation. - The CLI creates no bundle, kernel, root filesystem, tap, or share directory. Those are operator-supplied inputs. - Every command after process creation uses HTTP over the target VM's Unix socket. ### `flash run` Boot a VM from a bundle directory. The CLI starts a `flashvmm` process, waits for its socket, sends the create request, and normally waits for startup. Syntax: ```text flash run --bundle --disk-bps --disk-iops [OPTIONS] ``` Parameters: | Parameter | Required/default | Meaning | |---|---|---| | `--bundle ` | required | Directory containing the guest kernel and root filesystem. `flash` does not build bundles. | | `--id ` | generated if omitted | VM name and socket name. | | `--vcpus ` | default `1` | Number of guest vCPUs. Fixed for the VM's lifetime. | | `--mem-mib ` | default `512` | Guest RAM ceiling in MiB. Fixed by the KVM map at creation. | | `--mem-region-mib ` | optional | Attaches a virtio-mem device with a region of this size, MiB, reserved at creation. Absent means no virtio-mem device at all. The bundle's directory name must start with `mem-enabled-` — check with `flash bundle inspect`. | | `--cpu-millis ` | default `1000` | CPU budget in millicpu across all vCPUs. A value below `vcpus × 1000` throttles aggregate CPU time. | | `--disk-mib ` | default `512` | Disk capacity quota in MiB. | | `--disk-bps ` | required; minimum `1` | Disk throughput cap in bytes per second. Zero is rejected. | | `--disk-iops ` | required; minimum `1` | Disk operation cap per second. Zero is rejected. | | `--boot-grace-ms ` | platform default if omitted | Maximum uncapped boot interval. The CPU cap engages at the earlier of guest Start or grace expiry. Explicit `0` engages immediately. | | `--kernel-cmdline ` | optional | Extra kernel command line appended to machine-derived arguments. It does not replace required device-discovery arguments. | | `--nic ` | optional; repeatable | Attach one NIC. Format: `tap=NAME,mbps=N,pps=N,ingress-mbps=N` with optional `ip=CIDR,gateway=ADDR`. All three rate caps are required for every NIC. | | `--share ` | optional; maximum one | Attach virtio-9p: `host_dir=DIR,tag=TAG,at=PATH`. The host directory must already exist; `at` is the absolute guest mount path. | | `--entry ` | optional | Program or command to run inside the guest. Ignored for a warm spare created with `--pool`; the claim supplies the real entry. | | `--cpu-template ` | optional for run; required for future snapshot | CPU contract shown to the guest, such as `host-raw`, `t-amd-v1`, or `t-intel-v1`. | | `--vmm ` | default `flashvmm` | VMM executable. The CLI first supports a sibling binary and otherwise resolves this path. | | `--detach` | false by default | Return as soon as the VM is accepted instead of waiting for boot. | | `--pool` | false by default | Boot as a warm-pool spare, park at Ready, and wait for `flash claim`. | Example: ```bash flash run --bundle /opt/flashvm/bundles/current --id myvm \ --vcpus 2 --mem-mib 1024 --cpu-millis 1500 \ --disk-mib 2048 --disk-bps 104857600 --disk-iops 10000 \ --boot-grace-ms 8000 --cpu-template host-raw \ --entry /your/workload --detach ``` ### `flash restore` Rebuild the captured VM from a local snapshot or signed remote snapshot. Restore preserves the captured identity. Syntax: ```text flash restore (--from | --from-url --total ) \ --bundle --disk-bps --disk-iops [OPTIONS] ``` Parameters: | Parameter | Required/default | Meaning | |---|---|---| | `--from ` | local-source option | Snapshot file on this host. Conflicts with `--from-url`. | | `--from-url ` | remote-source option | Signed snapshot URL. Guest memory can fault in from it as pages are touched. Conflicts with `--from` and requires `--total`. | | `--total ` | required with `--from-url` | Total object size from the object listing. It bounds ranged fetches and is not learned by issuing a HEAD request to the same object. Conflicts with `--from`. | | `--bundle ` | required | Kernel and root filesystem corresponding to the snapshot. A snapshot carries device state, not the devices themselves. | | `--id ` | optional | Name for the restored VM. For restore, it must match the captured identity. | | `--vcpus ` | default `1` | vCPU count; must match the capture. | | `--mem-mib ` | default `512` | Guest memory in MiB; must match the capture. | | `--cpu-millis ` | default `1000` | Aggregate CPU budget in millicpu. | | `--disk-mib ` | default `512` | Disk quota in MiB. | | `--disk-bps ` | required; minimum `1` | Disk throughput cap in bytes per second. | | `--disk-iops ` | required; minimum `1` | Disk operation cap per second. | | `--nic ` | optional; repeatable | Rebuild a NIC from `tap=NAME,mbps=N,pps=N,ingress-mbps=N` plus optional `ip` and `gateway`. Devices come from this restore request, not the artifact. | | `--share ` | optional; maximum one | Rebuild the virtio-9p share from `host_dir=DIR,tag=TAG,at=PATH`. Omitting it restores with no share. | | `--cpu-template ` | optional argument | CPU template to present and validate. Use the template required by the snapshot's compatibility contract. | | `--vmm ` | default `flashvmm` | VMM executable. | | `--allow-bundle-mismatch` | false by default | Waive the normal refusal when the supplied bundle differs from the captured bundle. Use only when the change is deliberate and compatible with the captured memory state. | | `--detach` | false by default | Return as soon as restore is accepted. | Local example: ```bash flash restore --from /var/tmp/myvm.snap \ --bundle /opt/flashvm/bundles/current --id myvm \ --disk-bps 104857600 --disk-iops 10000 \ --cpu-template host-raw --detach ``` Remote example: ```bash flash restore --from-url '' --total \ --bundle /opt/flashvm/bundles/current --id myvm \ --disk-bps 104857600 --disk-iops 10000 \ --cpu-template host-raw --detach ``` ### `flash clone` Build a **new** VM identity from a snapshot while leaving the captured source identity conceptually separate. Clone accepts the complete `restore` parameter set and sends the intentional identity-mismatch permission. Parameters: `--from`, `--from-url`, `--total`, `--bundle`, `--id`, `--vcpus`, `--mem-mib`, `--cpu-millis`, `--disk-mib`, `--disk-bps`, `--disk-iops`, repeatable `--nic`, `--share`, `--cpu-template`, `--vmm`, `--allow-bundle-mismatch`, and `--detach`, with the same types, defaults, conflicts, and meanings documented for `restore`. Give every clone its own `--id` and network identity. A clone's MAC derives from the IP supplied in its NIC specification; duplicate IP configuration creates the same kind of collision as duplicate physical-host network configuration. Example: ```bash flash clone --from /var/tmp/myvm.snap \ --bundle /opt/flashvm/bundles/current --id replica-01 \ --disk-bps 104857600 --disk-iops 10000 \ --cpu-template host-raw \ --nic tap=tap1,ip=10.110.0.11/24,gateway=10.110.0.1,mbps=1000,pps=100000,ingress-mbps=1000 ``` ### `flash claim` Assign a real workload and reservation to a warm spare created with `flash run --pool`. Syntax: ```text flash claim --disk-mib --cpu-millis [OPTIONS] ``` Parameters: | Parameter | Required/default | Meaning | |---|---|---| | `` | required | Warm spare's VM name as shown by `flash ls`. | | `--entry ` | optional | Program to run. This becomes the actual workload; a spare's launch-time entry is ignored. | | `--entry-arg ` | default `0` | Integer entry argument delivered with the claim. | | `--disk-mib ` | required | Claimed workload's disk quota in MiB. | | `--cpu-millis ` | required | Claimed workload's CPU budget in millicpu. | | `--claim-id ` | optional | Claimed workload's reporting identity. Omit to keep the spare's VM ID. | Example: ```bash flash claim spare-01 --entry /your/workload \ --entry-arg 0 --disk-mib 2048 --cpu-millis 1000 --claim-id 101 ``` ### `flash ls` List registry entries and ask each reachable socket for current state. Dead endpoints are reported as stale; permission failures are inaccessible rather than stale. Parameters: only global options. ```bash flash ls flash --dir /run/flashvm ls ``` ### `flash logs` Read run logs, raw guest console, or the merged run-first view. Syntax: ```text flash logs [--stream run|console] [--since N] [--since-console N] [--limit N] ``` Parameters: | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--stream ` | no | Select only workload run output or only raw COM1 console. Omit for merged run-first output. | | `--since ` | no | Start from the run-log cursor returned as `next_since`. | | `--since-console ` | conditional | Start from the console cursor returned as `next_since_console`. Required beside `--since` for merged paging. | | `--limit ` | no | Maximum number of lines. | Examples: ```bash flash logs myvm --limit 100 flash logs myvm --stream run --since 100 --limit 100 flash logs myvm --stream console --since 50 --limit 100 flash logs myvm --since 100 --since-console 50 --limit 100 ``` ### `flash inspect` Show current state plus boot, CPU, memory, disk, NIC, share, and source configuration. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--nic ` | no | Include detailed state and tap byte counters for one NIC. | ```bash flash inspect myvm flash inspect myvm --nic 0 ``` ### `flash metrics` Read every measurable number in one point-in-time response. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--window ` | no | Maximum age accepted for guest-reported fields. Older guest values are returned as `null`, not presented as current. | ```bash flash metrics myvm flash metrics myvm --window 30 ``` The public metrics model combines guest CPU, memory, swap, and cache reports with host disk, quota, and NIC counters. Use a freshness window for health decisions. Use `flash watch --stream metrics` for continuous delivery. ### `flash watch` Follow a live SSE stream until interrupted or until an optional duration elapses. | Parameter | Required/default | Meaning | |---|---|---| | `` | required | VM name. | | `--stream ` | default `lifecycle` | `lifecycle`, `logs`, `metrics`, or `guest`. | | `--for ` | optional | Stop after this duration. Omit to run until interrupted or disconnected. | Lifecycle event names published by the CLI are `phase_changed`, `boot_completed`, `paused`, `resumed`, `paused_expired`, `source_expired`, `snapshot_completed`, `snapshot_failed`, `exited`, and `failed`. Examples: ```bash flash watch myvm flash watch myvm --stream metrics --for 60 flash watch myvm --stream logs --for 60 flash watch myvm --stream guest --for 60 ``` ### `flash pause` Freeze a guest for a bounded interval. A pause is never unbounded; this prevents an abandoned orchestrator operation from leaving a VM frozen indefinitely. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--max-ms ` | no | Maximum pause duration in milliseconds. A value below the safe guest-size floor is raised to that floor. | ```bash flash pause myvm --max-ms 5000 ``` ### `flash resume` Resume a paused guest. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | ```bash flash resume myvm ``` ### `flash stop` Stop the VM. Its process exits and its socket disappears. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | ```bash flash stop myvm ``` Use `flash ls` afterward to refresh inventory. `stop` returns a state-bearing reply; `rm` is the direct stop-and-remove operator intent. ### `flash set` Change supported live resource limits. Syntax: ```text flash set [CPU OPTIONS] [DISK OPTIONS] [NIC OPTIONS] ``` Parameters: | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--cpu-millis ` | no | Aggregate CPU budget in millicpu. vCPU count cannot be changed here. | | `--boot-grace-ms ` | no | Maximum uncapped boot interval. | | `--mem-mib ` | no, but always refused | Requests a memory-ceiling change. The ceiling is KVM-mapped at creation; use `balloon` for live reclaim, or `mem` for a VM created with `--mem-region-mib`. | | `--disk-mib ` | no | Disk quota in MiB. | | `--disk-bps ` | no | Disk throughput cap in bytes per second. | | `--disk-iops ` | no | Disk operations per second. | | `--nic ` | required for NIC rate changes | Select the NIC to which `--mbps`, `--pps`, and `--ingress-mbps` apply. | | `--mbps ` | no | Egress Mbit/s cap for selected NIC. `0` removes this cap. | | `--pps ` | no | Egress packet/s cap for selected NIC. `0` removes this cap. | | `--ingress-mbps ` | no | Ingress Mbit/s cap for selected NIC. `0` removes this cap. | Examples: ```bash flash set myvm --cpu-millis 750 --boot-grace-ms 0 flash set myvm --disk-mib 4096 --disk-bps 209715200 --disk-iops 20000 flash set myvm --nic 0 --mbps 500 --pps 50000 --ingress-mbps 500 flash set myvm --nic 0 --mbps 0 ``` ### `flash balloon` Ask the guest to hand pages back to the host, or read the last balloon request/result. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--target-pages ` | no | Target number of pages to reclaim. Omit to read the last request and actual result. | ```bash flash balloon myvm --target-pages 32768 flash balloon myvm ``` Ballooning is cooperative. The guest can return fewer pages than requested. A timeout is distinct from a completed request that freed zero pages. ### `flash mem` Set the live target for a virtio-mem region, or read its current state — size, block granularity, requested target, and what's actually plugged in. Only a VM created with `--mem-region-mib` has this device. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--target-mib ` | no | New live target for the region, MiB. Omit to read the current state. | ```bash flash mem myvm --target-mib 512 flash mem myvm ``` virtio-mem is host-driven, unlike the balloon: the host sets a real target and the guest's own driver grows or shrinks physical memory blocks to match it, live, with no reboot. A target beyond the region's own size is refused with `422 mem_exceeds_region` rather than silently capped. ### `flash snapshot` Capture a VM to a snapshot file. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--path ` | yes | Destination snapshot path. The VMM process must be able to write it. | ```bash flash snapshot myvm --path /var/tmp/myvm.snap ``` The VM must have been created with a CPU template. ### `flash snapshots ls` List snapshots created by one VM. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | ```bash flash snapshots ls myvm ``` ### `flash snapshots show` Show one snapshot's identity, size, and base. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `` | yes | Snapshot ID from `flash snapshots ls`. | ```bash flash snapshots show myvm ``` ### `flash snapshots rm` Delete one snapshot. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `` | yes | Snapshot ID from `flash snapshots ls`. | ```bash flash snapshots rm myvm ``` ### `flash nic add` Request attaching a host tap to an existing VM. This command is intentionally refused because the NIC's MMIO window and interrupt assignment are fixed when the VM is created and discovered by the guest. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--tap ` | yes | Host tap requested for attachment. | ```bash flash nic add myvm --tap tap1 ``` Correct action: put every required NIC in `flash run`, `flash restore`, or `flash clone` with repeatable `--nic` arguments. Use `flash set --nic` for live rate changes. ### `flash nic rm` Request detaching a NIC. This is refused because NIC topology is fixed for the VM's lifetime. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `` | yes | NIC index shown by `flash inspect`. | ```bash flash nic rm myvm 0 ``` ### `flash share show` Show the virtio-9p share, if configured. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | ```bash flash share show myvm ``` ### `flash share add` Request attaching a share to an existing VM. This is intentionally refused because the 9p device is built and described to the guest at creation. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--host-dir ` | yes | Existing host directory. | | `--tag ` | yes | 9p mount tag visible to the guest. | | `--guest-path ` | yes | Guest mount path. | ```bash flash share add myvm --host-dir /srv/data --tag data --guest-path /mnt/data ``` Correct action: use `--share host_dir=/srv/data,tag=data,at=/mnt/data` during `run`, `restore`, or `clone`. ### `flash share rm` Request detaching the virtio-9p share. This is refused because share topology is fixed. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | ```bash flash share rm myvm ``` ### `flash source show` Show the state of a remote snapshot source: identity, size, layout, and digests. The signed URL itself is not returned because it is a bearer credential. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name restored from a URL. | ```bash flash source show myvm ``` ### `flash source resign` Replace an expiring signed URL for the same remote snapshot without pausing the guest. The new object's identity, size, RAM offset, and chunk digests are checked before faults are redirected. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--url ` | yes | Freshly signed URL that must name the same snapshot bytes. | | `--total ` | yes | Total object size. | ```bash flash source resign myvm --url '' --total ``` ### `flash guest http` Send an HTTP-shaped request to the workload through the structured guest control channel. | Parameter | Required/default | Meaning | |---|---|---| | `` | required | VM name. | | `` | required | Workload path, such as `/health`. | | `--method ` | default `GET` | HTTP method carried to the workload. | | `--body ` | optional | Request body. The control frame has method, path, and body; arbitrary HTTP headers are not part of this command. | ```bash flash guest http myvm /health flash guest http myvm /jobs --method POST --body '{"job":"example"}' ``` ### `flash guest send` Deliver one topic and payload to the guest receive queue. | Parameter | Required/default | Meaning | |---|---|---| | `` | required | VM name. | | `` | required | Topic. The VM's configured allowlist decides whether the guest receives it. | | `` | required | Payload string. | | `--from ` | default `0` | Sender ID carried to the guest. | ```bash flash guest send myvm jobs '{"job":"example"}' --from 42 ``` ### `flash guest recv` Stream messages published by the guest. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | | `--for ` | no | Stop after this duration; otherwise run until interrupted or disconnected. | ```bash flash guest recv myvm --for 60 ``` This is the CLI counterpart to the guest-message SSE stream. The MCP stream name is `guest_messages`; the CLI `watch` stream name is `guest`. ### `flash version` Report one VM's VMM version and supported artifact formats. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | ```bash flash version myvm ``` Use `flash --version` for the CLI's version. ### `flash rm` Stop and remove a VM. The socket disappearing is the process-gone signal. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | VM name. | ```bash flash rm myvm ``` ### `flash bundle inspect` Report what a bundle directory declares about itself — kernel file and size, root filesystem presence and size, and which capability prefixes (`mem-enabled-`, `kvm-enabled-`) its directory name matches. Read-only, no VM socket involved; a way to check a bundle's capabilities before `flash run` rather than after. | Parameter | Required | Meaning | |---|---:|---| | `` | yes | Bundle directory to inspect. | ```bash flash bundle inspect /opt/flashvmm/bundles/mem-enabled-x86_64 ``` ### `flash reap` Remove registry sockets whose VMM process is gone. Reachability determines staleness; permission-denied sockets are inaccessible and are not deleted. Parameters: only global options. ```bash flash reap ``` ### `flash self-update` Inspect or install the current signed FlashVMM release. Auto-update is off until explicitly enabled. | Parameter | Required/default | Meaning | |---|---|---| | `--dry-run` | no | Report what would change without installing. | | `--status` | no | Compare installed and published versions without changing files. | | `--show-trust` | no | Print the release channel and trusted-key fingerprint compiled into the binary. | | `--install-timer` | no | Write and enable the systemd timer for unattended checks. Installation alone does not enable it. | | `--interval ` | default `15min` | Bare systemd time span used with `--install-timer`. | | `--force` | no | Override a refusal caused by package-manager ownership or machine-managed systemd ownership. Use deliberately because those systems may later restore their selected version. | Examples: ```bash sudo flash self-update --show-trust sudo flash self-update --status sudo flash self-update --dry-run sudo flash self-update sudo flash self-update --install-timer --interval 15min ``` The update verifies the signed manifest and artifact checksums before replacement. Running VMs continue using the executable image and opened bundle files they already have; new versions apply to subsequently started VMs. ## `flashnet` networking reference `flash` consumes a named tap but does not create one. `flashnet` is the host-networking helper for unattended setup. It performs privileged network configuration, starts the requested command through a child that has dropped all capabilities, and keeps the VMM itself outside the `CAP_NET_ADMIN` boundary. Syntax: ```text flashnet --mode --uplink [--vm-id ] -- [ARGS...] ``` Parameters: | Parameter | Required/default | Meaning | |---|---|---| | `--mode ` | required | Network mode: `bridge`, `nat`, or `routed`. | | `--uplink ` | required | Host interface used by the VM network. It may be Ethernet or Wi-Fi. | | `--vm-id ` | default `vm` | VM identity used by the networking workflow. | | `--` | required separator | Ends `flashnet` options; every following argument belongs to the child command. | | ` [ARGS...]` | required | Command launched after networking is ready, normally `flash run ...`. A `{NIC}` placeholder inside an argument is replaced with the generated NIC configuration. | Modes: | Mode | Behavior | Best fit | |---|---|---| | `bridge` | Connects the guest to the physical layer-2 segment. Guest DHCP configures the address. | Workloads that should appear directly on the upstream network. | | `nat` | Creates per-VM nftables NAT and uses a fixed `192.168.253.2/30` guest link. Exposure is explicit through port mapping. | Isolated outbound connectivity. | | `routed` | Acquires a real DHCP lease and makes it routable through proxy ARP without joining the physical layer-2 segment directly. | Routed guest identity with controlled host integration. | Example: ```bash flashnet --mode nat --uplink eth0 --vm-id demo -- \ flash run --bundle /opt/flashvm/bundles/current --id demo \ --disk-mib 256 --disk-bps 50000000 --disk-iops 5000 \ --nic '{NIC},mbps=100,pps=20000,ingress-mbps=100' \ --entry /mnt/demo/hello.sh ``` ## Architecture deep dive ### One process and one socket per VM The unit of isolation, control, placement, observation, and failure is a single FlashVM process. The `flash` CLI performs spawn + exec once at creation and uses HTTP for every subsequent operation. `flash-mcp` uses the same socket routes. `flashnet` performs the privileged host-network setup, drops capabilities in the child, and then launches `flash`. ```mermaid flowchart TB subgraph Host CLI["flash CLI"] MCP["flash-mcp
(MCP server, stdio JSON-RPC)"] NET["flashnet
(privileged network helper)"] subgraph VM_A["flashvmm process — VM A"] SockA["unix socket
/run/flashvm/A.sock"] VcpuA["vCPU threads"] DevA["virtio-blk · virtio-net · virtio-console
virtio-balloon · virtio-mem · virtio-rng
virtio-9p"] end subgraph VM_B["flashvmm process — VM B"] SockB["unix socket
/run/flashvm/B.sock"] VcpuB["vCPU threads"] DevB["virtio devices"] end end CLI -->|spawn + exec, once at creation| VM_A CLI -->|spawn + exec, once at creation| VM_B CLI -->|HTTP, every command| SockA CLI -->|HTTP, every command| SockB MCP -->|HTTP + SSE| SockA MCP -->|HTTP + SSE| SockB NET -->|creates TAP, drops caps, execs| CLI ``` Consequences: - There is no always-on management daemon that owns the fleet. - A process failure affects one VM rather than a shared VMM process. - Unix socket ownership and permissions are the local authorization boundary. - Fleet software can discover VMs from registry sockets and talk directly to each independent endpoint. - A standalone FlashVMM installation does not depend on another product repository, signing key, CDN, updater, dashboard, or cloud control plane to run. ### Warm-pool startup sequence A warm-pool spare performs kernel decompression, subsystem initialization, device discovery, guest-agent startup, and the Ready handshake before a workload needs it. It then parks until a claim arrives. Starting a workload becomes the act of assigning an entry point and reservation to an already-running guest. ```mermaid sequenceDiagram participant Host participant Kernel as Guest kernel participant Agent as Guest-agent (PID 1) Host->>Host: map guest memory (~700us) Host->>Kernel: load kernel image, build boot params
(ACPI DSDT + virtio-mmio on x86,
device-tree on aarch64) Kernel->>Kernel: decompress (LZ4), init subsystems
(the ~70ms+ of real boot time) Kernel->>Agent: exec /init (no separate init system) Agent->>Agent: mount pseudo-fs, start reader/broker/
metrics threads, spawn zombie reaper Agent->>Host: Ready (over COM2 control channel) Host->>Host: report.running() — flash inspect now
shows state:"running", boot_us stamped Host->>Agent: Start (with entry point) Agent->>Agent: exec the workload ``` Use `flash run --pool` to prepare the spare and `flash claim` to assign it. The claim can retarget disk capacity and CPU budget for the real workload. ### Snapshot, transfer, and cross-host restore FlashVMM composes migration from visible operations: capture the running VM, transfer the artifact, and restore it on the destination. There is no separate opaque migration service or dedicated `migrate` verb. ```mermaid sequenceDiagram participant HA as Host A participant File participant HB as Host B HA->>HA: POST /v1/flashvm/snapshots (running VM) HA->>File: write snapshot File->>HB: move HB->>HB: PUT /v1/flashvm (source: Snapshot) HB->>HB: CPU template checked, fail-closed Note over HB: refused if a promised
CPU feature is missing ``` CPU templates mask CPUID leaves and define the guest's CPU contract. Published template names are `host-raw`, `t-amd-v1`, and `t-intel-v1`. A destination that cannot supply a promised feature refuses the restore. The architecture supports both x86_64 and aarch64, GICv2 and GICv3 on ARM, bare metal and nested virtualization. ### MCP control topology Any MCP-speaking agent can launch `flash-mcp` locally. The server speaks stdio JSON-RPC to the agent and HTTP + SSE to each VM socket. ```mermaid flowchart LR subgraph Agent["Any MCP-speaking LLM"] C["Claude"] G["ChatGPT"] M["Gemini"] O["..."] end Agent -->|stdio JSON-RPC| MCP["flash-mcp"] MCP -->|HTTP + SSE| SockA["unix socket
/run/flashvm/A.sock"] MCP -->|HTTP + SSE| SockB["unix socket
/run/flashvm/B.sock"] SockA --> VMA["FlashVM A"] SockB --> VMB["FlashVM B"] ``` The agent uses the same lifecycle semantics as an operator: creation through the CLI, ongoing control through the socket, SSE for live context, snapshot for capture, restore for identity-preserving recovery or migration, and clone for a new identity. ## Virtio device model FlashVMM exposes exactly seven virtio device families: block, network, console, balloon, memory hotplug, random, and 9p shared filesystem. ### virtio-blk The default READ and WRITE path uses a 256-entry io_uring ring and batches submission to the host kernel. FLUSH, DISCARD, and WRITE_ZEROES remain synchronous. If the host cannot create the ring, the block device falls back to synchronous I/O automatically. Set `EMBER_BLK_IO_URING=0`, `off`, or `false` to force the synchronous path. ```mermaid flowchart LR Guest["Guest issues
READ/WRITE"] --> Ring["io_uring ring
(256 entries)"] Ring --> Batch["Batched submission
to the host kernel"] Batch --> Complete["Completions reaped,
accounted precisely"] Complete --> Guest2["Guest sees result"] Guest -.FLUSH/DISCARD/WRITE_ZEROES.-> Sync["Synchronous path,
always"] ``` Disk throughput and operation token buckets apply independently in this storage path. Disk capacity uses reserve-and-settle accounting based on allocated blocks plus a separate per-inode charge. ### virtio-net Define each interface at creation with: ```text --nic tap=NAME,mbps=N,pps=N,ingress-mbps=N[,ip=CIDR,gateway=ADDR] ``` The three rate controls are independent: - `mbps`: egress bandwidth. - `pps`: egress packet rate. - `ingress-mbps`: ingress bandwidth. All three are required and must be nonzero at creation. Repeat `--nic` for multiple interfaces. NIC topology is fixed after the VM is created; hot-add and hot-remove requests return `409 hotplug_unsupported`. Live rate caps can be updated with `flash set --nic `. A live value of zero removes that selected token bucket. ### virtio-console - COM1 is the raw kernel console and is exposed through `flash logs --stream console` and the console half of merged logs. - COM2 is the structured guest-agent control channel. It carries Ready and Start, guest metrics, workload logs, guest messages, and workload HTTP requests. ### virtio-balloon The KVM mapping fixes the RAM ceiling. A memory patch is refused with `422 memory_immutable`. The balloon asks the guest to return pages within its existing allocation. Always compare requested pages with actual freed pages; the guest may return fewer than requested. Treat a timeout differently from a completed zero-page result. ### virtio-mem A second, host-driven lever inside the same fixed ceiling. A VM created with `--mem-region-mib` reserves a memory region at creation, and from then on the host sets a live target with `PATCH /memory/mem` (or `flash mem --target-mib N`); the guest's own virtio-mem driver onlines or offlines physical blocks to converge on it. `GET /memory/mem` reports `region_mib`, `block_kib`, `requested_mib`, and `plugged_mib` — the actual, observable result, not just the last request. A target beyond the region's own size is refused with `422 mem_exceeds_region`, never silently capped. A VM created without a region answers both `GET`/`PATCH` with `422 mem_not_attached` and pays no cost for the device it doesn't have. Unlike the balloon, this is host-authoritative sizing, not guest-cooperative reclaim: set a target once, and the guest converges to it in fine-grained blocks — growing usable RAM into a workload's real footprint, or shrinking it back down, with no reboot and no resize cycle. ### virtio-rng The guest agent reseeds the guest kernel from virtio-rng on every boot. This protects cloned VMs from continuing with the same captured random-number-generator state. ### virtio-9p At most one host-directory share may be attached. Declare it at creation: ```text --share host_dir=DIR,tag=TAG,at=PATH ``` `host_dir` must already exist, `tag` is the mount tag delivered to the guest, and `at` is the absolute guest mount point. Share topology is immutable after creation. Restore and clone must supply the share again because a snapshot carries state, not the device definition. ## Networking and privilege separation The VMM does not hold `CAP_NET_ADMIN`. `flashnet` is the narrowly scoped helper that performs host mutations, opens the persistent tap, configures the selected mode, and hands execution to an unprivileged child. ```mermaid flowchart LR A["flashnet --mode {bridge|nat|routed}
--uplink IFACE"] --> B["Create persistent TAP device"] B --> C{mode} C -->|bridge| D["Guest joins physical L2 segment
guest DHCP configures itself"] C -->|nat| E["Per-VM nftables NAT table
fixed 192.168.253.2/30"] C -->|routed| F["Real DHCP lease via proxy-ARP"] D --> G["fork()"] E --> G F --> G G --> H["drop ALL capabilities
in the child"] H --> I["exec flash run ... --nic TAP"] I --> J["flashvmm: CAP_NET_ADMIN
never held, ever"] ``` Operational rules: - Create every tap before `flash run` consumes it. - Define all required NICs at creation. - Keep host network policy in `flashnet` or operator tooling, not inside the VMM. - Apply explicit per-NIC egress bandwidth, egress packet rate, and ingress bandwidth. - Use bridge for direct L2 presence, NAT for isolated outbound access, and routed mode for proxy-ARP-based routability. ## CPU scheduling and boot grace FlashVMM enforces CPU budgets without cgroups and without relying on voluntary guest exits. A pacer signals each vCPU thread every 5 ms. Because the signal handler does not use `SA_RESTART`, `KVM_RUN` returns `EINTR`; the vCPU thread then checks its `CLOCK_THREAD_CPUTIME_ID` consumption and sleeps when it has exhausted the budget window. ```mermaid sequenceDiagram participant Pacer as Pacer thread participant Vcpu as vCPU thread participant Budget as CpuBudget loop every 5ms Pacer->>Vcpu: tgkill(SIGUSR1), no SA_RESTART Vcpu-->>Vcpu: KVM_RUN forced to return EINTR Vcpu->>Budget: tick() — checked CLOCK_THREAD_CPUTIME_ID
against this thread's budget alt over budget Budget->>Vcpu: sleep until budget window resets end end ``` The VM fails closed if the signal handler cannot be installed. The default boot-grace interval is eight seconds. The CPU cap engages at the earlier of guest Start or grace expiry. `--boot-grace-ms 0` disables grace and engages the cap immediately. Restored VMs receive no boot grace. x86_64 and aarch64 use the same enforcement semantics. ## Streaming observability ### Data sources and SSE endpoints The guest agent reports CPU, memory, swap, and cache every three seconds. At each frame, the host adds disk, quota, and NIC counters. The VMM publishes metrics, logs, lifecycle events, and guest messages through per-VM SSE endpoints. ```mermaid flowchart LR Agent["Guest-agent
reports every 3s
(CPU/mem/swap/cache)"] --> Broadcast Host["Host-measured
disk/quota/NIC counters
(at frame arrival)"] --> Broadcast Broadcast["tokio::broadcast
channel"] --> SSE1["/metrics/events"] Broadcast --> SSE2["/logs/events"] Broadcast --> SSE3["/events (lifecycle)"] Broadcast --> SSE4["/guest/messages"] SSE1 --> Consumer["flash watch / flash_watch /
flash_watch_fleet"] SSE2 --> Consumer SSE3 --> Consumer SSE4 --> Consumer Broadcast -.slow consumer.-> Lagged["event: lagged
{missed: n}"] ``` Point-in-time and streaming choices: | Need | CLI | MCP | Endpoint | |---|---|---|---| | Complete current VM configuration/state | `flash inspect ` | `flash_inspect` | `GET /v1/flashvm` | | Current measurements | `flash metrics [--window N]` | `flash_metrics` | `GET /v1/flashvm/metrics` | | Combined diagnostic report | compose commands | `flash_diagnose` | inspect + metrics window + bounded logs | | Cursor-paged logs | `flash logs` | `flash_logs` | `GET /v1/flashvm/logs` | | Live metrics | `flash watch --stream metrics` | `flash_watch(stream="metrics")` | `/v1/flashvm/metrics/events` | | Live logs | `flash watch --stream logs` | `flash_watch(stream="logs")` | `/v1/flashvm/logs/events` | | Lifecycle | `flash watch --stream lifecycle` | `flash_watch(stream="lifecycle")` | `/v1/flashvm/events` | | Guest publications | `flash watch --stream guest` or `flash guest recv` | `flash_watch(stream="guest_messages")` | `/v1/flashvm/guest/messages` | | Same stream across several VMs | run bounded CLI watches | `flash_watch_fleet` | one SSE connection per VM | ### Log retention and cursor semantics Each VM keeps two 256 KiB buffers: one for the workload run log and one for the raw console. Within each buffer, the first 64 KiB is frozen and the latest 192 KiB rolls forward. When middle output is discarded, the returned data includes an explicit `--- N bytes dropped ---` marker and a `truncated` indicator. Log cursors are absolute line counts: 1. Begin without `since` or `since_console`. 2. Preserve `next_since` from the run portion. 3. Preserve `next_since_console` from the console portion. 4. Pass both back for the next merged page. 5. If reading only one stream, use the corresponding cursor with `stream=run` or `stream=console`. ### Backpressure and recovery If an SSE consumer falls behind the broadcast channel, the stream emits `event: lagged` with a missed count. Treat this as an explicit observation gap: 1. Record the gap and affected VM/stream. 2. Refresh current state with `flash_inspect` or `flash_resource`. 3. Refresh current measurements with `flash_metrics`. 4. For logs, resume from the last stored cursors and honor truncation markers. 5. Start a new bounded watch. ## Fleet scale and resource contracts FlashVMM treats each VM as an independent placement and enforcement unit. A fleet scheduler can select a host, prepare networking, create or restore the VM, apply limits, and observe it without coordinating through a shared local daemon. | Resource | Enforcement contract | |---|---| | CPU | Signal-based pacer using `tgkill`; no cgroup dependency. | | Memory | Fixed KVM mapping ceiling; balloon for live reclaim, virtio-mem for live host-driven growth and shrinkage of a region declared at creation. | | Disk capacity | Reserve before admission, settle after allocation, account allocated blocks and per-inode charge. | | Disk throughput | Token bucket in the block I/O path. | | Disk operations | Independent IOPS token bucket. | | Network egress bandwidth | Per-NIC token bucket. | | Network egress packet rate | Per-NIC token bucket independent of bandwidth. | | Network ingress bandwidth | Per-NIC token bucket. | | vCPU count | Fixed at cold boot and checked on restore. | | NIC count | Fixed at creation on x86_64 and aarch64. | Suggested fleet orchestration sequence: 1. Select a host with the required architecture, KVM availability, bundle, CPU-template compatibility, memory, disk, and network capacity. 2. If networking is needed, create it with `flashnet` or equivalent operator-owned host tooling. 3. Prefer a prepared warm spare when immediate assignment is the goal; otherwise run or restore. 4. Make all creation-time topology explicit: vCPUs, memory, NICs, share, CPU template, bundle, and entry point. 5. Make disk and NIC limits explicit. Never infer “unlimited” from a missing or zero creation-time limit. 6. Confirm the VM with `flash ls`/`flash_ls`, then inspect its state and configuration. 7. Use workload activity as the final readiness signal. 8. Stream lifecycle and metrics with bounded connections; partition fleets larger than 20 VMs across MCP fleet-watch calls. 9. Refresh point-in-time state after a lagged event. 10. Snapshot with a named CPU template, transfer the file, and restore on a compatible destination for migration. ARM support includes both GICv2 and GICv3. GICv2 restore covers distributor and per-vCPU interface state; private SGI/PPI state sits outside the published snapshot boundary. Nested virtualization follows the same lifecycle, including snapshot and restore. ## Security architecture ### Resource isolation FlashVMM does not assume a cooperative guest: - CPU time is interrupted and measured from the host. - Guest RAM cannot exceed the mapped KVM ceiling. - Disk allocation is reserved before writes are admitted. - Disk BPS and IOPS are controlled separately. - Each NIC controls egress bandwidth, egress packet rate, and ingress bandwidth separately. ### Process hardening order Hardening completes before the first guest instruction executes: ```mermaid sequenceDiagram participant CLI as flash CLI participant Proc as FlashVMM process CLI->>Proc: spawn + exec Proc->>Proc: drop all capabilities Proc->>Proc: no_new_privs, non-dumpable,
no core dump, fd ceiling Proc->>Proc: per-thread seccomp allowlist
(API thread, main thread, each vCPU) Note over Proc: guest code only
executes after this line ``` The API thread, main thread, and each vCPU thread receive their own seccomp allowlist in `KillThread` mode. A disallowed syscall terminates the issuing thread. The VMM also removes capabilities, sets `no_new_privs`, disables dumpability and core dumps, and applies a file-descriptor ceiling before guest execution. ### Socket authorization The control socket uses operating-system permissions rather than an additional application token: - Per-user registry directories are `0700`. - VM sockets are `0600`, or restricted to a configured group. - Inspect the effective policy with `ls -la` on the registry directory. - Run `flash-mcp` under an OS account that has access to the intended sockets. - Keep different tenants in different Unix users or carefully controlled groups. An extra token stored on the same host would not strengthen the boundary for a process that already has permission to read the socket and token. The filesystem policy is visible, auditable, and enforced by Linux. ### Network privilege boundary Only `flashnet` needs network-administration privilege for host setup. It creates and wires the tap, forks, removes capabilities from the child, and starts `flash`. The `flashvmm` process therefore does not inherit `CAP_NET_ADMIN`. ## Performance reference FlashVMM is faster than Firecracker on every published cold-boot path, ranging from approximately 5.4× to 29.2× faster across x86_64, aarch64, bare metal, and nested virtualization. ### Cold boot | Environment | FlashVMM | Firecracker | Published advantage | |---|---:|---:|---| | Local x86_64 bare metal | **72.6 ms** | 396.1 ms | FlashVMM ~5.5× faster | | Raspberry Pi 5, aarch64 | **8.65 ms** | 176.9 ms | FlashVMM ~20.4× faster | | AWS Graviton3 bare metal | **4.58 ms** | 133.7 ms | FlashVMM ~29.2× faster | | AWS nested virtualization, x86_64 | **126.9 ms** | 679.2 ms | FlashVMM ~5.4× faster | The published measurement endpoint is the moment the guest is ready and executing workload code. ### Snapshot restore | Environment | FlashVMM | Firecracker | Published result | |---|---:|---:|---| | Local x86_64 bare metal | **2.67 ms** | 8.9 ms | FlashVMM ~3.3× faster | | Raspberry Pi 5, aarch64 | 20.73 ms | 15.1 ms | Native GICv2 restore | | AWS Graviton3 bare metal | **1.36 ms** | 9.0 ms | FlashVMM ~6.6× faster | | AWS nested virtualization, x86_64 | **3.71 ms** | 10.1 ms | FlashVMM ~2.7× faster | The same lifecycle spans local hardware, cloud bare metal, edge systems, and nested virtualization, with GICv2 and GICv3 support on ARM. The public performance page also reports a successful 1,500-request sustained reliability run, batched io_uring storage, and synchronous capability removal before guest execution. ### Clone density The public Developer page reports that ten 512 MiB replicas created from one snapshot used 3.4% of the memory required by ten private copies and resumed in 15–37 ms. Clones share memory copy-on-write and resume from captured execution state instead of repeating cold boot. ## Troubleshooting and recovery playbook ### Installation completed but commands are missing or incomplete Validate all three layers: ```bash flash --version sudo flash self-update --show-trust ls -l /dev/kvm ``` If the installer reported success but the install is incomplete, rerun validation explicitly. Begin a new login session after adding the account to the `kvm` group. ### `cannot fetch …` The public troubleshooting guidance notes that this can represent a local write failure as well as a network problem. Check root filesystem capacity: ```bash df -h / ``` Then retry the install or update after resolving storage or connectivity. ### Manifest signature does not verify against this binary's trust anchor The installed binary trusts a different key from the release signature. Run the installation command again to establish the current trust anchor, then validate with: ```bash sudo flash self-update --show-trust ``` ### Permission denied on `/dev/kvm` Add the user to the KVM group and start a new login session: ```bash sudo usermod -aG kvm "$USER" ``` Do not solve ordinary VM operation by running the VMM through broad, permanent privilege. The public operating model is unprivileged VM commands plus the narrowly scoped `flashnet` helper for host-network mutations. ### No kernel in bundle Representative error: ```text no kernel in bundle '…' (looked for […]) ``` FlashVMM looks for a supported kernel filename such as `vmlinux`, `bzImage`, `Image`, or `kernel`. Point `--bundle` at the installed or operator-built directory containing both the kernel and root filesystem. ### Missing required disk flags or zero disk limit `run`, `restore`, and `clone` require both: ```text --disk-bps --disk-iops ``` Zero is rejected instead of being interpreted as unlimited. Choose explicit values based on the workload contract. ### VM cannot be captured Representative error: ```text capture refused: no CPU template was requested ``` CPU templates cannot be added to a running VM. Recreate it with at least: ```bash --cpu-template host-raw ``` Use a vendor template when the fleet requires a narrower compatibility contract. ### Restore identity mismatch Representative error: ```text restore refused [identity]: snapshot identity mismatch ``` Use the captured ID for `restore`. Use `clone` when the intent is a new identity: ```bash flash clone --from /var/tmp/myvm.snap --bundle \ --id new-id --disk-bps --disk-iops ``` ### Restore CPU contract mismatch The destination does not supply a CPU feature promised by the captured template. This is a fail-closed compatibility check. Select a compatible host or capture against a template designed for the intended fleet. ### Restore topology mismatch vCPU count and memory must match the captured topology. Reissue the restore with the correct `--vcpus` and `--mem-mib`. Recreate NICs and the share explicitly because the artifact contains device state but not the device definitions. ### Restore succeeds but a share is missing Add the original creation-time share to the restore or clone request: ```text --share host_dir=DIR,tag=TAG,at=PATH ``` The host directory must exist on the destination. A restore without `--share` has no share even though the guest control channel and workload state may otherwise be healthy. ### Share exists but does not mount in the guest Check console logs: ```bash flash logs --stream console ``` Confirm that the host directory exists, `at` is absolute, and the bundle's guest agent supports the configured mount tag. The console can report a failed 9p mount while the VM itself remains running. ### Memory update returns `422 memory_immutable` The KVM memory ceiling is fixed. Use the balloon for live reclaim: ```bash flash balloon --target-pages ``` Or, for a VM created with `--mem-region-mib`, resize its virtio-mem region live: ```bash flash mem --target-mib ``` Read the balloon or `flash mem` afterward to compare target and actual result. Recreate the VM if a larger ceiling — or a virtio-mem region at all — is required. ### NIC add or remove returns `409 hotplug_unsupported` NIC topology is a creation-time property. Declare every interface with repeatable `--nic` options during `run`, `restore`, or `clone`. Change live rate caps with: ```bash flash set --nic --mbps --pps --ingress-mbps ``` ### Network exists in the guest but traffic does not pass Work through the boundary in order: 1. Confirm `flashnet` ran with the intended `--mode`, `--uplink`, and `--vm-id`. 2. Confirm `{NIC}` was replaced or an explicit tap name was supplied. 3. Inspect NIC configuration and counters with `flash inspect --nic ` or `flash_resource(resource="nic", id="")`. 4. Confirm all three rate caps are nonzero at creation. 5. For bridge mode, confirm upstream L2 and guest DHCP. 6. For NAT mode, confirm the intended port exposure and per-VM NAT setup. 7. For routed mode, confirm the DHCP lease and proxy-ARP path. ### CPU update returns `503 not_ready` The boot-grace interval is active and the live-update path is not ready. Retry after boot or create the VM with immediate enforcement: ```text --boot-grace-ms 0 ``` ### CPU appears higher during early boot The default boot grace is eight seconds. The cap engages at the first of guest Start or grace expiry. Set a shorter interval or explicit zero at creation when the workload contract requires immediate enforcement. Restored VMs do not receive grace. ### No VM socket Representative error: ```text no VM at /run/flashvm/ID.sock — is it running? try `flash ls` ``` Run `flash ls` or `flash_ls` to refresh inventory. A VM process owns its socket, so an absent socket normally means the process is gone. If a dead process left a stale registry entry, run `flash reap`. Permission-denied entries are not stale and are not reaped. ### MCP tool cannot see a VM that the CLI can see Confirm that `flash-mcp` runs under the same OS user and therefore reads the same registry. Root uses `/run/flashvm`; ordinary users use `$XDG_RUNTIME_DIR/flashvm`. Confirm that `XDG_RUNTIME_DIR` is absolute and that socket permissions permit access. A CLI command using a custom `--dir` does not automatically change the registry used by the named MCP socket tools. ### `flash_cli` cannot find the CLI Set an explicit executable for the MCP server process: ```bash export FLASH_MCP_FLASH_BIN=/absolute/path/to/flash ``` Otherwise `flash-mcp` checks `/opt/flashvmm/bin/flash` and then `flash` on `PATH`. ### `flash_cli` times out `flash_cli` has a 120-second hard timeout. For creation, add `--detach` so the CLI returns after acceptance. For a long-running observation, use `flash_watch` with a bounded `for_secs` instead of invoking a CLI that waits indefinitely. ### `flash_watch` says a progress token is required The MCP request must include: ```json {"_meta":{"progressToken":"a-unique-token"}} ``` inside `params` beside `name` and `arguments`. This token identifies the `notifications/progress` stream associated with the outstanding tool call. ### MCP watch duration is refused `for_secs` must be between 1 and 3600. Break longer monitoring into bounded calls, refreshing point-in-time state between them. ### Too many concurrent MCP watches A `flash-mcp` process permits at most 20 active watches. `flash_watch_fleet` also permits at most 20 VM IDs per call. Partition a larger fleet across bounded calls or server processes according to the operator's connection budget. ### SSE stream reports `lagged` The consumer missed events. Record the missed count, refresh current state and metrics, resume logs from stored cursors, and start a new bounded watch. Do not infer missing state transitions from the remaining event sequence. ### Metrics are stale or null Guest-reported CPU, memory, swap, and cache arrive every three seconds. When a metrics freshness window is set, older guest values become `null`. This is intentional. Check lifecycle state, guest-agent health, console logs, and the live metrics stream rather than treating null as zero. ### Logs appear incomplete Check the `truncated` indicator and `--- N bytes dropped ---` marker. Each run/console buffer keeps the first 64 KiB and latest 192 KiB within a 256 KiB total. Preserve both cursors for merged paging and use live log streaming when continuous retention is required outside the VM. ### Snapshot operation status is needed Use one of: ```bash flash snapshots ls flash snapshots show ``` or MCP: ```text flash_resource(vm_id: "", resource: "operation", id: "") flash_resource(vm_id: "", resource: "snapshot", id: "") ``` Lifecycle streaming also reports `snapshot_completed` and `snapshot_failed`. ### Remote snapshot signed URL expires Inspect the source without exposing the bearer URL: ```bash flash source show ``` Then supply a fresh URL for the same object: ```bash flash source resign --url '' --total ``` The replacement is accepted only after the identity, size, layout, RAM offset, and chunk digests match. ## Operator and agent do/don't summary ### Do - Discover current IDs before taking action. - Use explicit CPU, memory, disk, and network contracts. - Select a CPU template at creation for any VM that may be captured. - Give every clone a unique VM and network identity. - Rebuild NICs and the 9p share on restore or clone. - Use point-in-time reads for decisions and SSE for continuous context. - Apply freshness windows to guest metrics. - Preserve both merged-log cursors. - Treat lag reporting as a real observation gap. - Bound watches by duration and connection count. - Run `flash-mcp` with the Unix permissions required for only the intended registry. - Use `flashnet` for host network setup so the VMM remains outside the network-admin privilege boundary. - Use ballooning for live memory reclaim, or virtio-mem for live growth and shrinkage of a region reserved at creation. - Use `clone` for a new identity and `restore` for the captured identity. - Verify update trust before enabling unattended updates. ### Don't - Do not invent defaults for required disk or NIC rate limits. - Do not interpret creation-time zero values as unlimited. - Do not create through raw socket HTTP. - Do not attempt to change vCPU count, memory ceiling, NIC count, or share topology after creation. - Do not omit the share or NIC definitions from restore when the workload needs them. - Do not waive a bundle mismatch without understanding the captured memory/root-filesystem relationship. - Do not expose signed source URLs through logs or status output. - Do not assume the first healthy-looking state is sufficient workload readiness. - Do not treat stale guest metrics as live data. - Do not discard `lagged`, truncation, or missed-count signals. - Do not give a single MCP server more than its 20-watch connection budget. - Do not run the VMM with network-administration capabilities. ## Frequently needed answers ### Is FlashVMM a container runtime? No. A FlashVM runs as a KVM microVM with its own guest kernel and virtio devices. The host process model remains lightweight—one VMM process per VM—but the isolation boundary is virtual hardware plus the guest kernel rather than a shared host kernel namespace. ### What host platforms are supported? The public product supports Linux on x86_64 and aarch64 with KVM. Published environments include local x86_64 bare metal, Raspberry Pi 5, AWS Graviton3 bare metal, and x86_64 nested virtualization. ### Does FlashVMM require a central daemon? No. The socket directory is the registry. Each VM has its own process and socket. Fleet controllers can be built above those independent endpoints without adding a mandatory host daemon. ### Can FlashVMM run without a cloud account or dashboard? Yes. Installation and operation use local binaries, Unix sockets, Linux permissions, and operator-provided artifacts. The public product is positioned as standalone, including air-gapped and self-hosted use. ### How are VMs created by an AI agent? The agent calls `flash_cli` with a real `flash run`, `restore`, or `clone` argv array. The CLI launches the VMM process and binds the socket. The agent then uses named MCP tools or `flash_api` for the existing VM. ### How does an AI agent collect metrics? - One VM, current sample: `flash_metrics(vm_id: "id")`. - One VM, current sample with explicit guest freshness: `flash_cli(args: ["metrics", "id", "--window", "30"])`. - One VM, diagnostic context: `flash_diagnose(vm_id: "id", metrics_window_secs: 30, log_lines: 100)`. - One VM, live stream: `flash_watch(vm_id: "id", stream: "metrics", for_secs: 60)` plus a progress token. - Several VMs, live stream: `flash_watch_fleet(vm_ids: [...], stream: "metrics", for_secs: 60)` plus a progress token, maximum 20 IDs. ### How does an AI agent collect logs? Use `flash_logs` for cursor-paged run/console data. Preserve `next_since` and `next_since_console` for merged mode. Use `flash_watch(stream: "logs")` for live delivery. Use `flash_diagnose` only for a bounded first-page excerpt, not as a true tail. ### How are snapshots used for migration? Capture the running VM, transfer the snapshot file to the destination host, and call `restore` there with the captured identity, compatible bundle, matching CPU/memory topology, required device definitions, resource limits, and CPU template. The destination refuses a CPU contract it cannot satisfy. ### What is the difference between restore and clone? Restore returns the captured VM and requires the captured identity. Clone creates a new VM from the artifact, allows a new identity, and needs its own network identity. ### How are rapid workload starts achieved? Create generic spares with `flash run --pool`. They complete kernel and device initialization and park at guest-agent Ready. `flash claim` assigns the entry point and reservation, turning the start into a claim rather than a boot. ### Can CPU, disk, and networking limits change while a VM runs? Yes. `flash set` can update CPU budget, boot grace, disk quota, disk throughput, disk operations, and per-NIC rate caps. vCPU count, memory ceiling, NIC topology, and share topology stay fixed. Ballooning reclaims pages within the memory ceiling, and a virtio-mem region — if the VM was created with one — resizes live, host-driven, within its own declared bound. ### How does FlashVMM handle slow telemetry consumers? The SSE broadcast pipeline emits an explicit `lagged` event with a missed count. Consumers should refresh point-in-time state and restart a bounded stream rather than silently assuming continuity. ### How are control permissions managed? Root sockets live under `/run/flashvm`; ordinary-user sockets live under `$XDG_RUNTIME_DIR/flashvm`. Registry and socket modes rely on Linux ownership and permissions. Run clients and `flash-mcp` under the intended user or group. ## Uninstall The public Developer page provides this removal sequence: ```bash sudo systemctl disable --now flashvm-self-update.timer sudo rm -rf /opt/flashvm /var/lib/flashvm /usr/local/bin/flash \ /etc/systemd/system/flashvm-self-update.{timer,service} \ /var/lib/systemd/timers/stamp-flashvm-self-update.timer sudo systemctl daemon-reload ``` This stops and removes the optional update timer, installed files, state, CLI link, and timer metadata. Review paths before executing removal on a customized installation. ## Compact glossary - **AF_UNIX / Unix socket:** local filesystem-addressed transport used by each VM API. - **Balloon:** cooperative virtio device that returns guest pages to the host within a fixed memory ceiling. - **virtio-mem:** host-driven virtio device that grows or shrinks a VM's usable RAM live, within a region reserved at creation, via a target the host sets rather than a request the guest cooperates with. - **Boot grace:** bounded interval during which the CPU limit is deferred; default eight seconds, zero for immediate enforcement, none after restore. - **Bundle:** directory containing the kernel and root filesystem used to build a VM. - **Claim:** assignment of a real workload and reservation to an already-booted warm spare. - **Clone:** new VM identity built from a snapshot. - **COM1:** raw guest kernel console. - **COM2:** structured guest-agent control channel. - **CPU template:** portable CPU feature contract checked at restore. - **FlashVM:** the KVM microVM managed by FlashVMM. - **GICv2/GICv3:** ARM generic interrupt-controller versions supported by snapshot and restore. - **io_uring:** Linux asynchronous I/O interface used for batched virtio-blk reads and writes. - **KVM:** Linux virtualization facility required by FlashVMM. - **MCP:** Model Context Protocol used by `flash-mcp` to expose FlashVMM to agents. - **Registry:** directory whose `.sock` entries identify per-VM control endpoints. - **Restore:** return the captured VM identity from a snapshot. - **SSE:** Server-Sent Events used for live metrics, logs, lifecycle events, and guest messages. - **Snapshot:** artifact containing captured execution and device state; device definitions are supplied again during restore/clone. - **Token bucket:** independent rate-enforcement mechanism for disk throughput, disk operations, and the three NIC traffic axes. - **Warm pool:** collection of prebooted spares parked at Ready until claimed. ## Final machine-actionable checklist Before create: ```text [ ] Linux x86_64 or aarch64 [ ] /dev/kvm accessible [ ] bundle path exists and contains kernel + root filesystem [ ] VM ID selected or generation accepted [ ] vCPU count selected [ ] memory ceiling selected [ ] CPU budget selected [ ] positive disk BPS selected [ ] positive disk IOPS selected [ ] disk quota selected [ ] CPU template selected if snapshot/restore/clone/migration may be needed [ ] every NIC and all three NIC caps selected [ ] at most one share selected and host directory exists [ ] entry point selected, unless creating a pool spare ``` After create: ```text [ ] refresh inventory with flash ls or flash_ls [ ] inspect configuration with flash inspect or flash_inspect [ ] confirm workload readiness [ ] collect freshness-bounded metrics [ ] start bounded lifecycle/metrics/log watches when needed [ ] preserve log cursors [ ] react to lagged events with state refresh ``` Before snapshot/restore: ```text [ ] CPU template was selected at creation [ ] snapshot destination is writable by the VMM [ ] restore identity matches capture, or clone is used for a new identity [ ] destination satisfies CPU template [ ] vCPU and memory topology match capture [ ] matching bundle supplied [ ] every required NIC supplied again [ ] required share supplied again [ ] positive disk BPS and IOPS supplied [ ] clone receives unique network identity ``` For MCP streaming: ```text [ ] _meta.progressToken supplied [ ] stream is metrics, logs, lifecycle, or guest_messages [ ] for_secs is between 1 and 3600 [ ] no more than 20 active watches per flash-mcp process [ ] no more than 20 VM IDs per flash_watch_fleet call [ ] VM ID prefixes retained for fleet attribution [ ] lagged notifications trigger point-in-time refresh ```