nix-mariner

NixOS microVM modules for creating development environments that isolate untrusted code from your host.

Built on microvm.nix.

What it does

Provides NixOS modules importable as a flake input.
Creates persistent microVM environments for isolating untrusted code away from the host.
Preconfigured with SSH, Docker, direnv, shared /nix/store, persistent storage, bridge networking.
Optional userlands inside the VM:
- Ubuntu via distrobox, SSH logins directly into Ubuntu.
- Android via Waydroid, opens a window on your host desktop.

Imperative and Declarative workflows

The documentation covers both imperative and declarative workflows. Before either, set up the host once: Host setup.

Imperative

Creates VM with microvm -c. The only host NixOS changes are the one-time Host setup.

See Imperative Virtual Machines.

Declarative

VMs defined inside the host’s NixOS configurations with microvm.vms.<name>.

See Declarative Virtual Machines.

Per-VM Customizations

You can change and override microvm.nix and nixos module configurations for each VM. Overrides work the same in both imperative and declarative modes.

See Customizing VMs.

Questions & Support

Feel free to start a discussion on Discussions
Report bugs and feature requests at Issues

Host setup

In order to use nix-mariner, you need to import mariner.nixosModules.host module and configure the network options in your nixos system configuration.

You can use the microvm cli tool to create and manage VMs imperatively, instead of declaring them in your NixOS config.

See Preparing a NixOS host for declarative MicroVMs for more information

NixOS microvm.nix module

Add mariner to your host NixOS flake inputs:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";

    mariner.url = "github:mksafavi/nix-mariner";
    mariner.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs =
    {
      nixpkgs,
      mariner,
    }:
    {
      nixosConfigurations.machine = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          mariner.nixosModules.host # Also imports the microvm host module
        ];
      };
    };
}

Alternatively, you could declare microvm directly in your inputs:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";

    microvm.url = "github:microvm-nix/microvm.nix";
    microvm.inputs.nixpkgs.follows = "nixpkgs";

    mariner.url = "github:mksafavi/nix-mariner";
    mariner.inputs.microvm.follows = "microvm";
    mariner.inputs.nixpkgs.follows = "nixpkgs";
  };

  outputs =
    {
      nixpkgs,
      mariner,
    }:
    {
      nixosConfigurations.machine = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          mariner.nixosModules.host # also imports the microvm host module
        ];
      };
    };
}

Mariner host module additions

mariner.host enables the microvm.nix microvm.host module and adds its own host-side options. You can enable optional features with:

  mariner.host = {
    enable = true;
    # Optional features:
    graphics.enable = true; # runs waypipe client to open windows on host compositor
    network.enable = true; # creates br-microvm bridge
    network.exposeDNS = true; # allows VMs to reach host DNS
  };

mariner.host.graphics runs a waypipe client so VMs can render windows on the host Wayland compositor.
mariner.host.network sets up a bridge that each VM connects to. It’s a default that might not match your setup. Keep it disabled and configure networking yourself if it doesn’t fit. See microvm.nix’s a simple network setup.
By default VMs resolve DNS themselves. mariner.host.network.exposeDNS allows the VM to use the host’s DNS server instead.

See host options for more information.

Verify

After a nixos-rebuild switch you should have the following:

ls /dev/kvm                        # exists
ip addr show br-microvm            # has 10.0.0.1/24
ss -lntp | grep ':53'              # listening on 10.0.0.1:53
lsmod | grep vhost_vsock           # module loaded

Imperative Virtual Machines

In the imperative workflow, you define virtual machines under nixosConfigurations.<vm> in a flake, then use the microvm CLI to manage their lifecycle. See Imperative MicroVMs for the upstream option reference.

Define a Virtual Machine in a Flake

Note

Before continuing, make sure you’ve completed the Host Setup.

Add nix-mariner as a flake input, then define a nixosConfigurations.<vm> entry for your virtual machine.

The following flake.nix creates a VM named example:

{
  inputs.mariner.url = "github:mksafavi/nix-mariner";

  outputs =
    { mariner }:
    let
      nixpkgs = mariner.inputs.nixpkgs;
    in
    {
      nixosConfigurations.example = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";
        modules = [
          {
            imports = [ mariner.nixosModules.default ];
            mariner.cid = 4; # Unique per-VM CID that sets vsock number and IP address.
            mariner.ssh.authorizedKeys = [ "ssh-ed25519 AAAA... your@host" ]; # Replace with your ssh public key
          }
        ];
      };
    };
}

Calling microvm -c builds the VM and creates the systemd service for booting it.

sudo microvm -c example -f path:$(pwd)

Verify that the vm is created:

microvm -l

Either start the service:

sudo systemctl start microvm@example.service

Or start it in foreground:

sudo systemctl start microvm-virtiofsd@example.service
sudo microvm -r example

[vm@nixos:~]$ poweroff # To exit the Virtual machine.

microvm -r runs the VM directly instead of through systemd, if you are using virtiofs (default) for shared storage you need to start the virtiofsd service first.

You can now ssh into it:

ssh vm@vsock%4
# or:
ssh vm@10.0.0.4

Declarative Virtual Machines

You can declare virtual machines directly in the host’s NixOS configurations by adding microvm.vms.<name> entries in a module.

nixos-rebuild switch then builds, updates, and starts them via systemd. You can’t modify VMs with microvm CLI anymore if you use the declarative workflow.

See Declarative MicroVMs for the upstream option reference.

Declare Multiple Virtual Machines

Note

Before continuing, make sure you’ve completed the Host Setup.

Create a virtualization.nix module and import it into your host setup:

{
  mariner,
  ...
}:
{
  microvm.vms = {
    vm-work = {
      config = {
        imports = [ mariner.nixosModules.default ];
        mariner.cid = 4;
        mariner.username = "work";
        mariner.ssh.authorizedKeys = [ "ssh-ed25519 AAAA... your@host" ];
      };
    };

    vm-test = {
      config = {
        imports = [ mariner.nixosModules.default ];
        mariner.cid = 5;
        mariner.username = "user";
        mariner.ssh.authorizedKeys = [ "ssh-ed25519 AAAA... your@host" ];
      };
    };
  };
}

Import the modules into your host configurations:

# In another module:
  imports = [
    modules/virtualization.nix
  ];

# Or in host flake:
  nixosConfigurations.machine = nixpkgs.lib.nixosSystem {
    #...
    modules = [
      modules/virtualization.nix
      # ...
    ];
  };

nixos-rebuild switch should build each VM and start the systemd services microvm@<name>.service

Now you can ssh into them if the services are running:

ssh work@vsock%4
ssh user@vsock%5

See microvm.autostart for starting the VMs automatically at host boot.

Per-VM Customizations

You can change and override microvm.nix and nixos module configurations for each VM. Overrides work the same in both imperative and declarative modes.

For more information, see Mariner options and microvm.nix Options.

{ mariner, ... }:
{
  config =
    { config, pkgs, ... }: # this points to the vm config to access config.mariner
    {
      imports = [ mariner.nixosModules.default ];

      mariner.cid = 3;
      # Change user name:
      mariner.username = "user";
      mariner.ssh.authorizedKeys = [ "ssh-ed25519 AAAA... your@host" ];
      # Set VM resources:
      microvm = {
        vcpu = 4;
        mem = 8 * 1024;
      };
      # Share a host directory with the VM:
      microvm.shares = [
        {
          source = "/home/you/work";
          mountPoint = "/work";
          tag = "work";
          proto = "virtiofs";
        }
      ];

      # Install more packages:
      users.users.${config.mariner.username}.packages = with pkgs; [ btop ];
    };
}

Storage and persistence

The root filesystem (/) and /tmp are tmpfs mounts that are wiped on every reboot. Anything you want to keep should be kept on a persitent volume like $HOME. Some modules provide persistent volumes and store them on the host as QEMU images under /var/lib/microvms/<name>/<image>.img

See options reference for the available volumes and sizes.

The host’s /nix/store is shared into the guest as a read-only filesystem, with a writable overlay on top of it. Set mariner.storage.readOnlyStoreShare to select the protocol for sharing the host’s store or set it to null if you don’t want to share the store.

Volume sizes

Volume images are sparse: a volume with an apparent size of 32GB only allocates the space that’s actually used. The allocated size will grow over time as you create and delete files, but a scheduled fstrim.service in the VM reclaims the freed blocks back to the host.

The size is only applied when the image is first created. Changing a SizeMiB option later doesn’t resize the volume.

$ du -hs --apparent-size  /var/lib/microvms/vm/*
32G     /var/lib/microvms/vm/nix-store.img
8.0G    /var/lib/microvms/vm/persist.img

$ du -hs  /var/lib/microvms/vm/*
1.4G    /var/lib/microvms/vm/nix-store.img
613M    /var/lib/microvms/vm/persist.img

Resizing a volume

You can resize the volumes after they are created. Make sure to stop the virtual machine before doing filesystem operations otherwise you will corrupt the volume and lose data. It’s recommended to only grow the images. For shrinking you have to first shrink the file system size before running qemu-img resize

sudo systemctl stop microvm@<name>.service

sudo qemu-img resize --format raw /var/lib/microvms/<name>/<image>.img <new-size>
sudo e2fsck -f /var/lib/microvms/<name>/<image>.img
sudo resize2fs /var/lib/microvms/<name>/<image>.img

sudo systemctl start microvm@<name>.service

Resizing persist volume to 64GB:

sudo systemctl stop microvm@vm.service

sudo qemu-img resize --format raw /var/lib/microvms/vm/persist.img 64G
sudo e2fsck -f /var/lib/microvms/vm/persist.img
sudo resize2fs /var/lib/microvms/vm/persist.img

sudo systemctl start microvm@vm.service

Clearing a VM

To clear a VM’s state, remove the appropriate image files in /var/lib/microvms/<name>/.

persist.img: $HOME and user data
nix-store.img: the writable store overlay
docker.img: the docker volume (only if mariner.docker.enable is set)
waydroid.img: the waydroid volume (only if mariner.waydroid.enable is set)

sudo systemctl stop microvm@<name>.service
sudo rm /var/lib/microvms/<name>/*.img

Linux distributions userland (distrobox)

You can run other Linux distributions’ userland inside the NixOS microVM, on the Docker backend. By default an Ubuntu box is configured, giving you an FHS environment where apt, .deb packages and systemctl just work. SSH drops you directly into the Ubuntu box.

Setting mariner.distrobox.enable = true; sets up ubuntu:24.04 with autoEnter configured:

{ ... }:
{
  mariner.cid = 4;
  mariner.ssh.authorizedKeys = [ "ssh-ed25519 AAAA... user@host" ];
  mariner.distrobox.enable = true;
}

Configuration

The manifest options module is freeform. Besides the documented options, any key the distrobox assemble manifest supports passes through to the generated distrobox.ini.

For more information, see Mariner options and distrobox assemble manifest

{ ... }: {
  mariner.distrobox.manifest.alpine = {
    image = "alpine:latest";
    additional_packages = [
      "git"
      "curl"
    ];
    # any assemble key works too, e.g.:
    # start_now = true;
  };
}

Limitations

Networking: The box is privileged but shares the VM’s network namespace (--network host), so networking is the VM’s responsibility, not the box’s. Manage the firewall, ports, routes and DNS from the NixosConfiguration, not from inside the box.
Kernel: The distrobox is sharing the VM’s kernel. Kernel modules load on the VM-side. modprobe in the box can’t work. If you need to add a module (e.g. a VPN’s tun), enable it on the VM.

What’s declarative or mutable

The distrobox manifest is declarative: which boxes exist, their image, hostname, additional_packages, and the autoEnter and replace behavior in declared in NixOS Configuration are reproducible.

The box’s filesystem is mutable: package installs and edits inside the box persist between reboots but are lost when the box container is rebuilt by making changes to the manifest. The box and its packages live on the docker.img volume. The user data is on $HOME on the persist.img volume, same as the NixOS VMs.

First boot

The box is built at first boot, not at Nix build time, so the first start of a box needs network access:

The distrobox-assemble service pulls the container image and creates the box, this might takes a while.
If you SSH in before the box is ready, autoEnter finds no box and drops you into the NixOS shell with a “not found” message.
On the first enter, distrobox installs base packages (plus your additional_packages) over the network.

Shell modes

By default SSH execs you into the box shell. To reach the VM’s NixOS shell without turning off autoEnter:

ssh -o SetEnv=MARINER_NO_AUTOENTER=1 vm@<addr>

Android emulation (Waydroid)

You can run Android inside a microVM via Waydroid. Enabling Waydroid enables graphics support it needs for rendering the Android UI window.

{ ... }:
{
  mariner.cid = 5;
  mariner.waydroid.enable = true;
  mariner.waydroid.systemImage = "GAPPS";
  mariner.ssh.authorizedKeys = [ "ssh-ed25519 AAAA... user@host" ];
}

Storage

Android /data(installed apps, their data, internal storage) is on the user’s $HOME on the persist volume. Increase the persist volume size if you need more storage space inside Android.

Viewing Android UI

Waydroid is running inside the VM and waypipe proxies the UI window to your host.

Guest

On the guest, waydroid-init.service initializes Waydroid and downloads the selected image on the first start. After that waydroid-session-start.service starts Waydroid session inside a waypipe server. Both steps run automatically on boot, so you don’t need to touch anything.

For troubleshooting, check the service logs:

journalctl -u waydroid-init
journalctl -u waydroid-session-start
sudo waydroid log

Host

You need to set mariner.host.graphics.enable to install the waypipe client service. It starts automatically with your graphical session:

systemctl status --user mariner-waypipe-client.service

If your compositor doesn’t manage a systemd graphical session, activate graphical-session.target or start the client manually.

waypipe --vsock -s 6000 client

And then:

ssh vm@<addr> waydroid show-full-ui

Network access on first boot

By default Waydroid downloads the Android system and vendor images on first initialization. After that the images persist on the waydroid volume and waydroid can start offline. You can also manually copy your images to /etc/waydroid-extra/images which will override the downloaded images.

Google Play Services

If you need Google Play store use the GAPPS image variant. You can also install a Google Play services package like LiteGapps using waydroid-helper.

run-waypipe waydroid-helper

For activating Google Play Store you need to certify the waydroid system with a Google Account at https://docs.waydro.id/faq/google-play-certification

Enable ARM64 to X86 translation layer

If you try to install an application that doesn’t publish X86 binaries you’ll get this error: not compatible with your device.

You need to install an ARM64 translation layer using waydroid-helper and clear Play store data. Install libhoudini for Intel machines or libndk for AMD. Installing both will crash Waydroid at boot.

Known limitations

No audio: Audio is not configured yet
Blank window flashes: When the Waydroid session is starting you might see blank windows opening for a split second, mostly a cosmetic issue.
Turning off or Restarting Waydroid doesn’t restart the session. you need to manually restart waydroid-session-start.service
Android network is not reachable from LAN/host.
Manual images directory is not persisted on the waydroid volume yet.

Caveats

Don’t run `nix-collect-garbage` inside a VM

A microVM mounts the host’s /nix/store as read-only (/nix/.ro-store) and layers a writable overlay on top. Running nix-collect-garbage inside the VM cannot delete anything from the read-only base, instead it records whiteout deletions in the writable overlay.

So from the VM’s perspective every path in /nix/.ro-store can be marked deleted, while the host store stays untouched. The downside of this is that you lose the shared store paths but the VM should still operate correctly.

nix-collect-garbage will whiteout delete every unreferenced paths from the read-only store:

[vm@nixos:~]$ nix-collect-garbage --dry-run
finding garbage collector roots...
determining live/dead paths...
123791 store paths would be deleted

The running VM system is protected, because /nix/var is mounted early and its closure is registered as a valid GC root.

Recovery: If you ran nix-collect-garbage and lost access to the read-only store you can recover it by deleting the writable store overlay volume image (/var/lib/microvms/<name>/nix-store.img) and restarting the VM. The read-only host store will show up cleanly again. Your data/persist volumes are separate and aren’t affected.

Mariner options reference ⚙️

mariner.cid

VSOCK context ID

Type: integer between 3 and 254 (both inclusive)

Declared by:

/modules/vm.nix

mariner.distrobox.enable

Whether to enable distrobox integration.

Type: boolean

Default:

false

Example:

true

Declared by:

/modules/distrobox.nix

mariner.distrobox.autoEnter

Automatically enter distrobox on interactive login

Type: null or string

Default:

"ubuntu"

Declared by:

/modules/distrobox.nix

mariner.distrobox.manifest

Generates distrobox assemble manifest.ini, each attribute name is a [section] for a box. Freeform Option: you may add any key that assemble manifest supports. For the full list see: distrobox-assemble manifest reference.

Type: attribute set of (open submodule of section of an INI file (attrs of INI atom (null, bool, int, float or string) or a list of them for duplicate keys))

Default:

{
  ubuntu = {
    image = "ubuntu:24.04";
  };
}

Declared by:

/modules/distrobox.nix

mariner.distrobox.manifest.<name>.additional_packages

Extra packages to install in the box

Type: list of string

Default:

[ ]

Example:

[
  "git"
  "curl"
  "vim"
]

Declared by:

/modules/distrobox.nix

mariner.distrobox.manifest.<name>.entry

Generate desktop entry. Must be false on headless distrobox

Type: boolean

Default:

false

Declared by:

/modules/distrobox.nix

mariner.distrobox.manifest.<name>.hostname

distrobox host name

Type: string

Default:

"‹name›"

Declared by:

/modules/distrobox.nix

mariner.distrobox.manifest.<name>.image

Which image should the distrobox container use

Type: string

Declared by:

/modules/distrobox.nix

mariner.distrobox.manifest.<name>.init

Run systemd init as PID 1 inside distrobox

Type: boolean

Default:

true

Declared by:

/modules/distrobox.nix

mariner.distrobox.replace

Whether to recreate boxes. always on every boot, never only manually with distrobox assemble create --replace or onChange when the manifest file changes

Type: one of “onChange”, “never”, “always”

Default:

"onChange"

Declared by:

/modules/distrobox.nix

mariner.docker.enable

Whether to enable Docker integration.

Type: boolean

Default:

false

Example:

true

Declared by:

/modules/docker.nix

mariner.graphics.enable

Whether to enable Graphical desktop apps support.

Type: boolean

Default:

false

Example:

true

Declared by:

/modules/graphics.nix

mariner.network.enable

Whether to enable guest LAN interface.

Type: boolean

Default:

true

Declared by:

/modules/network.nix

mariner.network.address

Static IPv4 address assigned to the LAN interface.

Type: string

Default:

"Derived from `mariner.cid`"

Declared by:

/modules/network.nix

mariner.network.dns

DNS Address defaults to FallbackDNS. You should set it to network.gateway if host.network.exposeDNS is enabled.

Type: null or string

Default:

null

Declared by:

/modules/network.nix

mariner.network.gateway

Host static IPv4 address on the bridge.

Type: string

Default:

"10.0.0.1"

Declared by:

/modules/network.nix

mariner.network.mac

MAC address for the LAN interface.

Type: string

Default:

"Derived from `mariner.cid`"

Declared by:

/modules/network.nix

mariner.ssh.authorizedKeys

A list of SSH authorized public keys for vm user and root

Type: list of (optionally newline-terminated) single-line string

Default:

[ ]

Declared by:

/modules/ssh.nix

mariner.storage.dockerSizeMiB

Size of the docker volume in MiB. Stores Docker containers, images and volumes. Only created when mariner.docker.enable is set.

Type: null or (positive integer, meaning >0)

Default:

null

Declared by:

/modules/storage.nix

mariner.storage.nixStoreSizeMiB

Size of the writable Nix store overlay in MiB. A writable overlay on the read-only host nix store, caches nix-shell and flake outputs built inside the VM.

Type: null or (positive integer, meaning >0)

Default:

null

Declared by:

/modules/storage.nix

mariner.storage.persistSizeMiB

Size of the /persist volume in MiB. Holds the user’s $HOME and anything written under /persist in the VM.

Type: null or (positive integer, meaning >0)

Default:

Declared by:

/modules/storage.nix

mariner.storage.readOnlyStoreShare

Sets the protocol for sharing the host machine /nix/store as a read-only share. setting it to null disables it. Check the microvm.hypervisor supported protocols before setting this option.

virtiofs performs better than 9p and requires microvm-virtiofsd@<name>.service to run on the host besides the VM which is automatically started as a dependency of microvm@<name>.service.

Type: null or one of “virtiofs”, “9p”

Default: