> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mirage.strukto.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Watch Matrix

> TypeScript watch delivery support and the pull or push detection paths available for each VFS.

## How To Read This Matrix

Watching has two independent halves:

* **Delivery** (`ws.watch()` and `ws.notify()`) works on **every mount**.
  Mirage invalidates caches, matches scopes, and delivers the event stream.
* **Detection** is backend-specific. **Pull** marks mounts whose
  `deltaHook()` ships with Mirage. **Push signal** names the provider mechanism
  that your application can map to a `FileEvent`.

| VFS | Pull (`deltaHook`) | Fingerprint | Push signal (provider-side) |
| - | - | - | - |
| Nextcloud | ✓ recursive PROPFIND | WebDAV ETag | `webhook_listeners` app (Nextcloud 30+) |
| S3 / S3-compatible | ✓ recursive `ListObjectsV2` | object ETag | S3 Event Notifications (SQS/EventBridge) |
| GitHub | ✓ recursive git tree | blob sha | repository webhooks |
| Dropbox | ✓ `list_folder/continue` (walk is reset) | `content_hash` | [`list_folder/longpoll` + cursor delta](https://github.com/strukto-ai/mirage/blob/main/examples/typescript/dropbox/watch.ts) |
| GridFS | ✓ `fs.files` prefix query | revision ObjectId | MongoDB change streams |
| Hugging Face | ✓ recursive tree listing | Hub ETag | — |
| Disk | ✓ recursive directory walk | mtime\|size | inotify / FSEvents, `DiskEventHook` |
| SSH / SFTP | ✓ per-directory descent | mtime\|size | — |
| Google Drive | ✓ per-folder descent | `modifiedTime` | `changes.watch` push channels |
| OneDrive / SharePoint | ✓ per-folder descent | `cTag` | Graph change notifications + delta API |
| Box | ✓ `/events` stream (walk is reset) | content `sha1` | Box `/events` long poll, `BoxEventHook` |
| Slack | — | — | Events API, mapped by `SlackEventHook` |
| Redis | — | — | keyspace notifications, `RedisEventHook` |
| RAM, others | — | — | no external writers to observe |

The **Fingerprint** column is what decides UPDATE. A content-addressed one
(GitHub's blob sha, S3's single-part ETag, Dropbox's `content_hash`, Box's
`sha1`) reports
nothing when a write stores identical bytes. `mtime|size` cannot: rewriting a
file with its own contents moves the mtime, so disk and SSH report an UPDATE.
That is the filesystem's own resolution, not a Mirage choice.

### Cost Per Pull

Not every hook costs the same, and the poll cadence should follow the shape:

* **One request per pull**, whatever the tree: GitHub (one recursive tree
  call), Nextcloud (one recursive PROPFIND), Hugging Face, and GridFS (one
  prefix query). S3 is one request per 1000 keys.
* **One request per directory**: Google Drive and OneDrive/SharePoint key
  their trees by opaque id and offer no whole-subtree listing, so the walk
  descends folder by folder. SFTP does too, for the same reason. Box has the
  same tree, but its pull reads the `/events` stream instead: an idle pull is
  one request, and the per-folder walk only runs as the reset.

Where a provider offers a native cursor (Dropbox `list_folder/continue`, Graph
`/delta`, Drive `changes.list`), that is a **faster** pull, not a more correct
one, and it does not replace the walk: a server may invalidate a cursor at any
time, and the only answer to that is a full listing. Those fast paths belong
behind `pull()` with the walk as their reset.

## Delivery Without a Hook

Any backend becomes watchable as soon as your application has a signal:

```ts theme={null}
import { FileChangeKind, FileEvent, PathSpec } from "@struktoai/mirage-node";

// your consumer, on any mount, today
for await (const event of ws.watch("/s3/exports/*.csv")) {
  console.log(event.kind, event.path.virtual);
}

// your S3 SQS consumer, your GitHub webhook route, your cron diff:
await ws.notify(
  new FileEvent({
    kind: FileChangeKind.CREATE,
    path: PathSpec.fromStrPath("/s3/exports/jan.csv"),
    timestamp: new Date(),
  }),
);
```

`watch()` returns an async generator, so it subscribes on the first iteration,
not at the call: the loop above and the `notify` call belong to different tasks
(a request handler, a poll loop). Awaiting `notify` before anything consumes
the iterator drops the event.

The guarantee is the same on every backend: caches for the changed path and
its ancestor listings are invalidated before delivery, plus the cached subtree
of a removed folder that still has a listing cached at or under it, so reads
after an event are fresh within the limit [Watch](/typescript/watch) describes.

One thing every backend has to agree on for that to hold: the index is
keyed by the mount-absolute path (`/m/data/x`), which is what
`CacheManager` builds when it evicts. GitHub was the one exception, keying
its by the repo-relative path because its index is a materialized git
tree, and the mismatch was silent: evicting a key an index never held
succeeds, so a GitHub mount kept serving pre-change bytes with nothing
failing anywhere a caller could see. It now keys like the rest, and the
repo-relative path logic that wanted the other spelling (`find`, `du`,
grep's scope counter) reads the git tree on the accessor instead, which is
where a repo-relative path belongs.

## Adding Pull Detection

`ListingDeltaHook` provides a generic checkpointed diff. A backend supplies an
async walk that reads the backend directly and yields stable fingerprints:

```ts theme={null}
import {
  ListingDeltaHook,
  type PathSpec,
  type WalkEntry,
} from "@struktoai/mirage-core";

async function* walk(root: PathSpec): AsyncGenerator<WalkEntry> {
  for await (const object of listObjects(root.vfsPath)) {
    yield {
      virtual: `/s3/${object.key}`,
      isDir: false,
      fingerprint: object.etag,
      size: object.size,
      modified: object.lastModified,
    };
  }
}

const hook = new ListingDeltaHook(walk);
```

Two shared helpers cover the shapes that recur, so a new backend usually
writes neither loop itself:

* **`synthDirs`** builds the directory rows a prefix store implies but does not
  store. An object store has no directories, so a walk that reported only keys
  would show a file appearing inside a directory that never appeared. S3 and
  GridFS use it, and it takes explicitly stored markers too, so an empty
  directory made by `mkdir` is still reported.
* **`ReaddirWalk`** covers a backend with no recursive listing at all. It
  descends through the backend's own `readdir` and `stat`, exactly as `find`
  does, and gives each pull a **fresh private index**. That is what keeps the
  DeltaHook contract: the index is not Mirage's read cache, so the walk cannot
  compare the cache to itself, and it starts empty every pull. It has to exist
  at all because Drive, Box and Graph resolve a path's id through the index its
  parent's readdir populated; a null index makes every path below the root read
  as absent.

The [Dropbox longpoll example](https://github.com/strukto-ai/mirage/blob/main/examples/typescript/dropbox/watch.ts) owns the loop in the embedding
program: establish a baseline with `pull`, wait for `list_folder/longpoll`,
then pull and `notify` each change. No EventHook or library background loop is
needed. The example extracts the cursor from the **latest** v1 checkpoint
(`_dbx: 1`, `c`), while passing the entire checkpoint back to `pull`. This is
Dropbox-specific example code; other consumers should treat checkpoints as opaque.
On reset, pull with the old checkpoint so the existing hook can relist and diff
against its saved snapshot. A missing watch root retries after 30 seconds.

Set `DROPBOX_APP_KEY`, `DROPBOX_APP_SECRET`, and `DROPBOX_REFRESH_TOKEN`;
optionally set `DROPBOX_ROOT_PATH`. From `typescript/`, run
`pnpm --filter @struktoai/mirage-examples exec tsx dropbox/watch.ts`. Ctrl-C
stops polling and finishes closing the workspace. Because `DeltaHook.pull`
has no cancellation parameter, this standalone example then allows one more
second before exiting with status 130 if an in-flight pull still holds the
process open. The deadline never interrupts workspace cleanup.

Dropbox's [longpoll endpoint](https://docs.dropboxapi.com/dropbox-api/api-reference/user-endpoints/files/list-folder-longpoll)
uses the notification host without an Authorization header. Its timeout is
30–480 seconds; the HTTP timeout must also allow up to 90 seconds of server
jitter. The returned `backoff` is a required minimum delay before the next
longpoll, including a response with no changes. The examples let transport or
unexpected API failures surface and close their resources on cancellation.

Dropbox `pull` already uses `list_folder/continue` and keeps the listing walk
as the reset path. Box `pull` reads the account's `/events` stream from a
stored `stream_position` and places each event through its `path_collection`.
Box never refuses an old position and may repeat an event or send it out of
order, so the tree is walked again once the last walk is two weeks old, and
whenever an event moves the watch root itself or a folder above it. Other
backends with a native cursor can do the same. The opaque checkpoint can be a
Dropbox cursor, a Microsoft Graph delta link, or any other serialized provider
state. Keep the walk as the reset path, though. A cursor is a promise the
server keeps, and when it breaks (`path/reset`, `resyncRequired`) a full
listing is the only answer.

A walk that knows it saw only part of the tree must throw
`IncompleteWalkError` rather than return what it has. A snapshot diff reads
every unlisted path as a DELETE, so a partial listing does not degrade into
fewer events, it invents wrong ones. GitHub does this when the API truncates a
large repository's tree.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.