Networking stack

The FlashVMM networking stack gives each FlashVM up to eight virtio-net NICs, wired on Linux by flashnet in bridge, NAT or routed mode on host taps (and pre-built in NIC pools by flashnetd), and on macOS and Windows by a built-in user-mode NAT, with per-NIC token-bucket rate caps that change on a running VM.

The VMM process leaves host networking to dedicated tools: on Linux it runs without CAP_NET_ADMIN and opens taps that already exist. Network setup is a separate, root-owned job (flashnet for one VM, flashnetd for pools), so the process that runs guest code stays unprivileged. Every frame passes through the VMM, which is where rate caps, the egress engine and the secret relay act.

Key facts

How it works

The guest NIC

Each NIC is a modern (v2) virtio-mmio virtio-net device with one RX and one TX queue, advertising VIRTIO_F_VERSION_1 and VIRTIO_NET_F_MAC, so the datapath is a straight frame copy. The guest's MAC is derived from the four octets of the NIC's assigned IP. The guest agent configures every addressed NIC at boot, before the workload starts; only NIC 0's gateway becomes the default route. On Linux the guest's TX kick reaches the network pump through a KVM ioeventfd, without a VM exit.

Rate control

Each NIC carries optional token buckets: egress bytes (mbps), egress packets (pps) and ingress bytes (ingress-mbps). A cap applies only when given. flash set ID --nic N --mbps … or PATCH /v1/flashvm/nics/{nic_index} re-rates a running NIC on every OS; 0 deletes that bucket.

Linux: flashnet modes

flashnet creates a tap, wires it to an uplink, runs the given command with {NIC} replaced by the resulting --nic value, and undoes every step in reverse order when the VM exits (or on SIGINT, SIGTERM, SIGHUP). All wiring is native netlink and nftables.

• NAT. The VM gets a /30 carved from 10.200.0.0/13 by hashing its --vm-id, so a restore of the same VM gets the same subnet its guest already holds. flashnet refuses a /30 already routed elsewhere. Masquerade lives in a dedicated nftables table per tap. The tap's MAC is derived from the gateway address, so a restored guest's cached gateway entry stays valid. The uplink defaults to the interface carrying the default route.
• Bridge. The tap joins a Linux bridge on the uplink; the guest sits on the LAN and runs its own DHCP client. The bridge takes the uplink's MAC and IPv6 settings, is shared by every flashnet VM on that uplink under a lock, and the host's network is put back when its last user exits.
• Routed. flashnet obtains a LAN address for the guest by DHCP through a short-lived macvlan probe, installs a /32 host route to the tap and answers for the guest with proxy ARP.

Linux: NIC pools

flashnetd is one root daemon on /run/flashnetd.sock that builds NICs before any VM starts. A pool is up to 1,000 NICs of one mode, each built by the same per-mode setup as a single flashnet run, with taps named fnp<id>-<n> and owned by the UID allowed to claim them. flash run --nic pool=ID (also fork and rewind) claims one; it returns to the pool when its VMM lets go of the tap (the tap loses carrier), when nothing opens it within 30 s, or on release, and it is rebuilt clean: NAT and routed NICs get a new address and fresh firewall state, bridge NICs cycle their link. A pool can carry a default egress policy for every VM that claims from it. Pool state is on disk, so a restarted daemon re-adopts its NICs. See pools.

macOS and Windows: built-in user-mode NAT

Each OS uses its native unprivileged path: the HVF engine on macOS and the WHP engine on Windows exchange Ethernet frames with a bundled user-mode NAT process over a pipe, so a FlashVM reaches the network with no host privilege and no host interface changes. The guest's default segment is 192.168.127.0/24; the gateway at .1 also answers DNS, and .254 maps to the host's loopback, which is where the secret relay listens. The Virtualization.framework engine on macOS uses the same datapath and rate caps.

Interfaces

CLI

# Linux: one VM on NAT; flashnet fills in {NIC} and tears it all down on exit
sudo flashnet --mode nat --vm-id web1 -- \
  flash run --bundle ./bundle --vcpus 2 --mem-mib 1024 --id web1 --nic {NIC},mbps=100

# Linux: a pool of 50 NAT NICs owned by UID 1000, then claim one
sudo flashnet pool create --mode nat --count 50 --id web --owner 1000
flash run --bundle ./bundle --vcpus 2 --mem-mib 1024 --nic pool=web,mbps=200
flashnet pool ls
flashnet pool show web

# Any OS: caps at create, then re-rate live
flash run --bundle ./bundle --vcpus 2 --mem-mib 1024 --id app \
  --nic tap=t0,mbps=100,pps=10000,ingress-mbps=100
flash set app --nic 0 --mbps 500 --ingress-mbps 0

MCP

flash_cli(args: ["set", "app", "--nic", "0", "--mbps", "500"])
flash_api(vm_id: "app", method: "GET", path: "/v1/flashvm/nics/0")

REST

GET   /v1/flashvm/nics
GET   /v1/flashvm/nics/{nic_index}
PATCH /v1/flashvm/nics/{nic_index}           {"mbps": 500, "pps": 0, "ingress_mbps": 100}

flashnetd (/run/flashnetd.sock):
POST   /v1/nic-pools                         {"id": "web", "mode": "nat", "count": 50}
GET    /v1/nic-pools
GET    /v1/nic-pools/{pool_id}
DELETE /v1/nic-pools/{pool_id}
POST   /v1/nic-pools/{pool_id}/claims        {"vm_id": "app"}  -> 201 {"tap", "nic"}
DELETE /v1/nic-pools/{pool_id}/claims/{tap}

NIC byte counters are the tap's, from the host's perspective (rx_bytes is what the host received from the guest), and are omitted when the tap cannot be read. Contract:

• PATCH with none of the rate fields returns 422 nothing_to_change; an index the VM lacks returns 404 unknown_nic.
• A NIC in the launch config whose tap failed to open returns 409 nic_not_attached.
• NICs are fixed at boot: POST /v1/flashvm/nics and DELETE /v1/flashvm/nics/{nic_index} return 409 hotplug_unsupported. To take a NIC out of service, re-rate it.
• Pool ids are 1 to 6 characters of a-z0-9 (they name the taps).

Guarantees

• An unprivileged VMM. Host network changes are made by a separate root tool; the process running guest code opens a ready tap or pipe.
• Clean teardown. Every flashnet step is undone in reverse, including on signals and partial failures.
• Stable across restore. A restored VM gets back the NAT subnet and gateway MAC its guest already holds.
• Clean reuse. A pooled NIC is rebuilt before its next VM gets it.
• Live control. Rate caps and egress policy change on a running NIC.

See networking, NIC pools and egress control.