grep, cat, pipes, redirects) itself; python3,
node and every other program run on the machine that captures them.
A runtime is an interpreter that runs inside
Mirage, on an
in_process machine: Monty, WASI, Pyodide or QuickJS. A host or
a provider takes no runtime; it runs the programs installed on it.
Routing
machines is ordered: the first machine whose captures name a program runs
it, and Mirage runs the rest itself. Without the key, python3, node and
js answer command not found (exit 127) until a machine captures them.
provider: in_process, host, remote or a
provider. The entry’s other keys are its settings, snake_case in
YAML for both SDKs; a key it does not take is refused.
With two @external machines the first one runs every program, and the
second runs what it names on its own:
execute(machine=), a
machine on the shell request, or a policy’s Route(machine=): see
Route Machine. A named machine runs what it
captures; everything else stays where it would run. When a machine takes
@line, a line that names a different machine does not run whole on the
@line machine: the named machine runs what it captures, and each other
command runs where its own capture sends it, or in Mirage. On a line the
@line machine runs whole, a command that a pre_command policy routes to
another machine capturing it exits 126.
Captures
captures lists the programs a machine takes:
- Mirage’s own commands (
cat,grep,sha1sum), pipes and redirects stay in Mirage. Namecat,grep,psorkillto run it on a machine instead;cd,export,echo,ln, functions and installed CLIs stay in Mirage even when named. @externaldoes not takepython,python3,nodeorjs: Mirage has those itself and gives them to the Python or JavaScript machine in the list. Name them, or@pythonor@js, to send them somewhere else. A tool such aspipis an ordinary program: capture it on the same machine when packages must land there, as incaptures: ["@python", "pip", "pip3"].- Monty, WASI, Pyodide and QuickJS take only their own interpreter’s
names; naming another program on one is refused when the workspace is
built. A
hostmachine and a provider take any name. - Mirage matches the program each command really runs, after expanding its
words, so
python3is captured in$P x,env python3 x,timeout 5 python3 x,sh -c 'python3 x'andxargs python3. Each command on a line matches on its own, socat a.txt | python3 xsends onlypython3. nice,nohup,stdbuf,ioniceandsetsidrun their program where it runs alone:nice gcc x.cwheregccis taken,nice grepin Mirage, and permission rules and apre_commandroute answer for the program. The process the program runs as carries the change:psshows its niceness, anohupjob ignoresHUP, asetsidone leads its own group and session. Ahostmachine or a provider is handed the wrapper with the program, so it makes the change there.- An absolute path Mirage has no file for, such as
/usr/bin/ffmpeg, goes to the@externalmachine. A profile’scommandsjudges it by its name, so under"*": denyit runs only when the tree names it:ffmpeg: allowlets/usr/bin/ffmpegrun. - On a
hostmachine or a provider, a program a captured program starts runs there:makerunninggcc, or a script callingsubprocess. Capture the launcher on the same machine as the tool. A Pyodide script’ssubprocesscalls go back through Mirage’s routing like any other command. - An alias is captured as the word it expands to, once
shopt -s expand_aliasesis on and the alias was defined on an earlier line. - If two machines take the same name or group, the first wins and Mirage logs a warning.
- A program no machine takes exits 127,
command not found. capturesonly places programs. To limit what a session may run at all, use its profile’scommands; see Command rules.- A name only
@externaltakes has no file, sowhich,command -vandtypereport it missing; running it still works. - Globs expand against the workspace first (
python3 job.py *.csvrunspython3 job.py a.csv b.csv), and permission rules judge that argv.
python3 job.py | grep error runs python3 in the container and
grep in Mirage, and ffmpeg -i a.mp4 b.webm runs in the container.
Files
Mount modes, profiles and policies apply to every file call that reaches the
workspace. A program on a
host machine or a provider also has that machine’s
own files and network, which they do not cover. Pyodide has its own
containment limits; use a machine with a process and kernel boundary for
hostile code.
If a host machine or a provider cannot enforce the session’s file
restrictions, Mirage refuses to launch the program as a policy refuses a
command: <provider>: Permission denied, exit 126, naming a host machine by
its own name. See ssh for the restrictions a provider cannot enforce.
Verify workspace file access
Run these lines inside Mirage’s shell after configuring a writable/data
mount visible to the machine capturing python3. They check that Python can
read a workspace file and write one back:
host machine, first mount a FUSE folder.
For a provider, finish its page’s Connecting the workspace steps first.
The machine’s own files
A workspace with machines shows the machines’ own/tmp and /dev/shm, and
/tmp: "@gpu" picks one machine’s. A file Mirage writes there is the file
the machine’s programs read:
Providers
A provider reaches a machine you already have: a container, a microVM, a cloud sandbox or a host over SSH. You create and stop it; Mirage only runs programs on it. Itscaptures default to @external, every
program Mirage does not have itself.
To try one, start with Docker.
Connecting the workspace
A program on a machine opens workspace files at the same paths as in Mirage, so the machine must see the mounts.mount_type picks how:
Daytona takes no
mount_type: run a Mirage on the machine, as own does,
or put the files there yourself. boxd takes none either: put the files
there yourself. Once connected, run the shared
workspace file check.
ssh
- The machine needs only
sshfs. No Mirage, keys or open port. - Each file call is a round trip to the workspace. With
both, keep bulk data on the machine’s own mounts. - A machine shows one session’s view, the first that runs a line there. An
sshfs already serving a path Mirage serves is left alone and the line exits
- Give each agent and each workspace its own machine.
- If the machine restarts, Mirage mounts the paths again. A line caught mid-restart exits 126 without running.
- A write the workspace refuses, such as one past a mount’s
capacity, looks done to the program, but its command fails when the program ends: stderr names the file, asdocker: /data/big.bin: Disk quota exceeded, and a command that would exit 0 exits 1. - The root mount and
/devare not served, nor a path the machine holds itself:/tmpand/dev/shmshow the machine’s own files (Machine Files), so they stay its own.
/, /dev or a path the machine holds itself, hiding a mount when that would expose
the machine’s own files underneath, restrictions on mounts a Mirage on the
machine serves, and rules that depend on which program opens a file.
mirage
-
The machine needs FUSE (
fuse3) andmirage-fuse, found in~/.mirage/binor on itsPATH. When it has none, Mirage installs it into~/.mirage/binat its own version, withuvornpm, whichever the machine has. No Mirage, keys or open port. -
Files on
rammounts, and onsqlmounts whose database only the workspace opens (one in memory, a Durable Object’s own), stay cached on the machine until they change, so a program that walks a project, such aspytestorgit status, runs at about local-disk speed after its first read. A change made through Mirage reaches the machine before its next program starts. -
Every other mount behaves as under
ssh: the machine keeps an answer for a second. -
While a policy judges the session’s file calls (
pre_vfsorpost_vfs), neithermirage-fusenor the machine’s kernel keeps an answer, not even for a second, so each call a program makes reaches it. A policy added later drops what the machine kept before the next program starts, and a file a program opened from the cache before then can no longer be read (Permission denied). The machine asks only for what a program opens or lists, never ahead of it. -
Sessions, restarts, refused writes and the paths left unserved work as
under
ssh.
mirage-fuse runs on Linux, x86_64, arm64 or 32-bit ARM (armv7 and armv6,
so every Raspberry Pi), with glibc or musl, and on macOS, Apple silicon or
Intel. For a machine without uv, npm or network access, put it in the
image with either:
mirage-fuse --version ends with fsproto 1.
A Mac
A Mac as a machine is experimental: its setup and behavior may still
change.
mount_type: mirage, not ssh. mirage-fuse mounts through
macFUSE, which the Mac’s owner installs once, its
system extension allowed; Mirage cannot install it.
-
A prefix mounts at its own path when the machine user can make that
folder (
/Users/me/project,/tmp/work). -
macOS keeps its root folder read-only, so a prefix there (
/data) needs a link in/etc/synthetic.confinto Mirage’s folder,~/.mirage/mnt/data. Mirage makes it withsudo -nwhere it may; otherwise the line stops and names the two commands an administrator runs once: -
Programs,
ps,kill,timeout,topandfreework as on Linux. macOS hides the environment of its own programs (/bin/sh,/usr/bin/*), so Mirage knows a run there by its program and what runs under it; a program that leaves its run and is one of Apple’s is no longer shown or stopped with it.
own
Build the machine image, write the same mounts at the same paths intosandbox.yaml, and start Mirage with it on the machine. Each
provider page has the commands.
- This workspace’s rules do not reach those files, so a line is refused while the session’s permissions hide or narrow anything under them.
- If a path the machine mounted goes away, lines exit 126 until it is mounted again. Start Mirage from the image’s entrypoint so a restart mounts it again.
- Keep the machine’s paths in step with the host’s mounts yourself.
sandbox.yamlcan differ where it should, such as narrower credentials.
What a program sees
Processes and jobs
One session can run jobs on several machines at once.jobs, wait, kill
and ps ask the machines what is running, so they answer with what is true
there now. ps lists each program with its real pid and its machine, and what
the program started under it:
ps -o pid,machine selects those columns, and ps f draws each command
under its parent’s. $! is the program’s real pid, through nohup and other
wrappers too, and STAT is what the
machine reports, T for a program kill -STOP paused.
kill sends a signal to a program as you sent it: a program that traps USR1
runs on, and one that TERM ends exits with its own status, which wait
reports. KILL is sent only when you ask for it (kill -9, timeout -k) or
when Mirage shuts the workspace down.
Mirage’s own commands have pids from 4194305 up. A pid two machines both have
is refused as an ambiguous job spec, so name the machine with -m or
--machine, as ps shows it or without the provider: kill -m box 13 or
wait --machine ssh:gpu 13. All of this follows the session’s processes
grants.
pgrep, pkill, pidof and killall search the processes ps shows, so they
reach every machine at once and do not see processes Mirage did not start.
Reading machines
hostname, nproc, lscpu and free answer for the workspace, as mirage,
and for each machine. uptime and top -b answer for the machines. One answer
prints what the real command prints. Several print a row each with the machine
on the right, so a column you read by position stays where it is; hostname
prints one name a line. Figures are never added up, as one program cannot use another machine’s memory.
With one machine, free answers for that machine alone, in the real free’s
table, and free --machine mirage shows the workspace.
free,
its max_memory with what it holds of that. It
prints - for everything else, so nothing about the server Mirage runs on shows. lscpu leaves the flags and
vulnerability rows out of its table; lscpu --machine cloud prints that
machine’s full output, and any other option goes to the default machine’s own
lscpu. df and mount list the workspace’s mounts, not a machine’s disks.
Inside $( ) they answer for the machine programs run on by default, so
make -j$(nproc) gets one number. --machine picks another:
workers=$(nproc --machine cloud). --machine mirage picks the workspace. The
host counts as a machine only while a host entry runs programs on it, and prints
- for figures it has no such thing for. uptime, top and lscpu exist only
in a workspace that runs programs on a machine or the host; there top needs
-b, as it does without a terminal.
ps -o %cpu,%mem,rss,vsz,time shows what each program has used, and
ps --machine cloud narrows it to one machine. top reads each
machine’s processors twice a second apart, so it takes about a second.
Machine states
Mirage finds a run’s processes by
MIRAGE_PROCESS in /proc, so it misses
one that drops the variable or whose /proc entry it cannot read, such as a
process of another user.
GET /v1/workspaces/{id}/machines checks every machine, and
GET /v1/workspaces/{id}/machines/{machine} checks one and lists its
processes. {machine} is the name ps shows, provider:name such as
docker:box; a bare name answers 400. Each machine answers its type, its
name and its id, what its provider calls it, such as the sandbox id.
mirage workspace list-machines -w ID prints each machine, its id and its
state.
Pause
workspace.pause() closes the workspace as close() does, then pauses each
machine whose config asks. What a pause does is the provider’s:
Mirage pauses only a machine the workspace reached, and a machine that is
already gone counts as paused. A pause interrupts anything else running on the
machine, so leave
on_pause unset on a machine others share.
To choose at pause time instead, name the machines to pause, whatever their
config says; every other machine is only disconnected:
close() first refuses pause(), since its machines were left running.
The machine image
Every machine needs a POSIX shell at/bin/sh and grep. E2B and Daytona
also need dd and base64, which carry a program’s output back. With
libfaketime in the image, programs
see the workspace’s system_time. With
mount_type: ssh the machine also needs sshfs, with mount_type: mirage
fuse3 (macFUSE on a Mac) and uv or npm to install mirage-fuse, and a
machine that runs its own Mirage needs fuse3 and Mirage.
The repository’s Dockerfile builds an image with fuse3, Mirage and
libfaketime for every provider. It has no sshfs: add it for
mount_type: ssh.
MIRAGE_EXTRAS for every mountable backend. Add the programs your
captures need, such as Node.js for node.
Timeouts
command_limits apply to every machine’s programs as to any command: a line
past its @line entry’s timeout_seconds exits 124 (line: timed out after Ns), and max_bytes and max_lines cap a program’s output.
/bin/sh, tr,
grep and /proc on the machine.
A dropped SSH connection with a terminal sends HUP
instead, then CONT, with no KILL after it: a process that ignores HUP
keeps running. On Linux it reaches only the processes still in the run’s
session, so one that left with setsid keeps running too.
Without a terminal the drop sends nothing until the run writes, as sshd’s
broken pipe would end it, and then sends the same HUP.