cd /data; wc -l notes.txt is one line. For each line,
Mirage records the line, its exit code, the first 4096 characters of its
stdout and stderr, and the reads and writes it made to a mount, such as wc
reading a file. Mirage does not record every mount call yet. A stat, a
directory listing, a delete or a move may leave no record.
Mirage keeps these records as command logs and serves them back to you, to
the agent and to policies. If your process sets up an OpenTelemetry SDK,
Mirage also sends each line to your tracing backend as a trace.
A session runs a line
↓→
Mirage
records it
↓→
Command logs
kept by Mirage in memory or on disk, read with
history, /.bash_history, mirage workspace logs or ws.history()Traces
sent through your OpenTelemetry SDK to a backend such as Datadog, Honeycomb, Grafana or Langfuse
Logs or traces
Use command logs when the agent, a policy or you need to look back at what a
workspace ran. Use traces to watch many machines in one place, with your
tracing tool’s search, retention and alerts.
Example
This config mounts a RAM folder at/data.
demo.yaml
demo from it, and run three lines in it.
The log entry
In a terminal,mirage workspace logs -w demo prints a table of lines, newest
first. Piped, it prints a JSON page whose items hold one entry per line.
mirage workspace logs -w demo | jq '.items[1]' prints the entry for the
second line. cwd is where the line started. mounts
has the read that wc made. The cd is not in mounts, because checking
that /data is a directory is a lookup.
cat has "exit_code": 1 and the error in
stderr. Its cwd is /data, because the session kept the cd from the
line before.
The raw records
Underneath, Mirage stores each line as records linked byparent: a
command record when the line starts, a vfs record for each recorded
mount call, and a result record when it ends.
mirage workspace logs -w demo --jsonl prints them, oldest first:
await ws.history(vfs=True) in Python, or await ws.history({ vfs: true })
in TypeScript, returns the same records, with timestamp in epoch
milliseconds instead of a date string.
The trace
If the process that runs the workspace has an OpenTelemetry SDK set up (see Trace setup), the same line also becomes one trace with two spans. Theshell span covers the whole line, and the process span sits
under it. The file wc read stays in the command log: a line’s calls to
mounts are its commands’ own work.
ws.vfs or a
FUSE mount, is a vfs.<call> span of its own,
such as vfs.read:
shell span carries the line’s id as mirage.line.id, so you can
match a trace to its log entry. The span below comes from a separate run, so
its ids differ from the entry above.
Reading logs
mirage shell --no-record runs a line without logging it.
Log entries
Mirage splits input into lines the same way bash does, and logs one entry per line.
A
set -e failure stops a line the same way exit does. If a
post_execute policy checks a line, Mirage writes
the entry after the policy answers. Output that the policy drops never
reaches the logs.
Log storage
By default, a workspace you build in code keeps its logs in memory. A workspace the server builds keeps them on disk under~/.mirage/state.
To choose, set store.logs to one of these backends.
store.logs lets the logs live apart from the rest of the workspace state.
This config keeps the state on Redis and the logs on local disk.
- YAML
- Python
- TypeScript
store.logs, the logs go where store.type points. Redis does not
hold logs, so with store.type: redis they go to disk under the store’s
root.
Restarts
Each log belongs to a session. A rebuilt workspace shows a session’s logs only after it has that session again. To keep logs across a restart, do one of these.- Keep the sessions on a store that survives a restart.
- Give the default session a fixed id with
default_session_id.
Trace setup
Mirage sends nothing until your process sets up an OpenTelemetry SDK. Mirage depends only on the OpenTelemetry API, so install the SDK and an exporter yourself. The exporter picks the backend. Set up the SDK before the workspace runs its first line.- Python
- TypeScript
Set up the SDK in code.Or run without code changes.
agent.py
opentelemetry-instrument sets up the
SDK for you.agent.py
Join a caller’s trace
Send the caller’s W3C trace context with a line and itsshell span goes
under the caller’s span, so your service’s spans and Mirage’s land in one
trace. Each access carries it the standard way:
tracestate travels the same way. A line without a trace context starts
a trace of its own. In TypeScript, the process running the
workspace needs a context manager and the W3C propagator; NodeSDK
registers both.
Spans
Each line’s spans form one tree: theshell span and everything it
started. A process sits under the process that started it, and a background
job’s process under its job span. Use the span and attribute names in your
backend’s queries, dashboards and alerts.
All spans and attributes
All spans and attributes
Span attributes
Traces cover what runs in the workspace and on its machines. A program that a
machine runs on another host sends no spans of its own.
Trace limits
Next steps
- See every
storesetting,logsincluded, in the configuration reference. - Check lines before or after they run with policies, which read
/.bash_history. - See every
mirage workspace logsflag in the CLI reference.