> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mirage.strukto.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# HF Datasets

> Mount a Hugging Face Dataset repo as a Mirage filesystem in TypeScript.

`HfDatasetsVFS` mounts a [Hugging Face Dataset](https://huggingface.co/datasets)
repo at a prefix such as `/ds/`. All reads are lazy: only the bytes you actually
`cat`/`head` get transferred. The TypeScript backend mirrors the
[Python one](/python/vfs/hf_datasets) and returns identical results.

For credential setup, see [HF Datasets Setup](/home/setup/hf_datasets).

<Note>
  Node only: the VFS ships in `@struktoai/mirage-node`. It needs no native
  binding, though, because it speaks the Hub API over the runtime's own `fetch`.
  The [opendal](https://www.npmjs.com/package/opendal) binding belongs to
  [HF Buckets](/typescript/setup/hf_buckets), which is a different product.
</Note>

## Node

```bash theme={null}
pnpm add @struktoai/mirage-node
```

```ts theme={null}
import { HfDatasetsVFS, MountMode, Workspace } from '@struktoai/mirage-node'

const dataset = new HfDatasetsVFS({
  repoId: 'AlienKevin/SWE-ZERO-12M-trajectories', // "namespace/dataset-name"
  token: process.env.HF_TOKEN, // optional for public datasets
  // Optional:
  // endpoint: 'https://huggingface.co',
  // revision: 'main',
  // keyPrefix: 'train/',
})

const ws = new Workspace({ '/ds/': dataset }, { mode: MountMode.READ })

console.log((await ws.shell('ls -lh /ds/')).stdoutText)
console.log((await ws.shell("find /ds/ -name '*.parquet' | wc -l")).stdoutText)
console.log((await ws.shell('cat /ds/README.md | head -n 20')).stdoutText)
```

`HfDatasetsVFS` takes `repoId` in `namespace/dataset-name` form plus an
optional access token. Public datasets need no token.

## Reading, not writing

This mount is read-only, the way a [`github`](/typescript/setup/github) mount is. A Hub write is
a **commit**, and a POSIX write cannot say where a commit ends, so `echo >`,
`rm`, `cp` and `mv` are refused here rather than silently making one commit
per file. The [`hf` CLI](/typescript/cli/hf) is the write half: `hf download --local-dir`
puts a copy on a ram or disk mount, which is an ordinary writable filesystem,
and `hf upload` sends it back as a single commit.

## Listings under `fresh`

Under `read: fresh`, a cached listing is checked against the commit the
`revision` resolves to: one small check per command
(`revision/{rev}?expand[]=sha`), after which every cached listing of the
mount at that commit is served. A mount whose `revision` is a full 40- or
64-hex commit sha is checked the same way, with that one small request
per command: a branch or tag named like the sha could take the name, and
mirage does not assume which one the Hub resolves. With a
`keyPrefix`, the stored version is that commit joined with the prefix, so mounts
of different subtrees sharing an index never share a version. See
[listings under `fresh`](/home/cache#listings-under-fresh).

## Filesystem Layout

Maps dataset repo files (README, parquet shards, etc.) to virtual paths under
the mount prefix. Parquet shards stream lazily via byte-range reads.

## Shell Commands

Every read command in [HF Buckets'](/typescript/setup/hf_buckets#read-commands) set works here, as
do the text processing and path utilities, which only read. What does not is
the [File Operations](/typescript/setup/hf_buckets#file-operations) group: this mount is read-only,
so `rm` and `touch` are refused, as is any command asked to write into it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.