# miracache: quick start

> Source: https://github.com/mirastacklabs-ai/miracache/blob/main/README.md (public repo, ref main), sections "Requirements" and "Quick start"; pulled at build time 2026-11-19. The repo has no docs/quickstart.md yet; this file switches to it automatically when it appears.

## Requirements

| Role | Needs |
|---|---|
| Memory node | x86-64 Linux (tested on Ubuntu 24.04), an RDMA NIC speaking RoCEv2 or iWARP (or soft-RoCE / soft-iWARP for trials), hugepages, plenty of DRAM. Three or more nodes |
| GPU host | vLLM 0.32–0.34 (accepted range; contract-tested at 0.33.0) or SGLang 0.5.23 (contract-tested), Python 3.10+, the same RDMA fabric as the memory nodes |
| Build | clang-18, CMake 3.28+, Ninja, rdma-core (`libibverbs-dev`, `librdmacm-dev`), OpenSSL (`libssl-dev`). Go is fetched and checksum-verified by `deploy/docker/go.sh` |

## Quick start

**1. Build**

```sh
cmake --preset release && cmake --build --preset release   # mira-memnode, mira-nicprobe, libmiracache.so
deploy/docker/go.sh -C controller build -o "$PWD/build/release/bin/" ./cmd/mira-controller ./cmd/mirctl
# or container images: deploy/docker/build.sh
```

**2. Check each host**

```sh
build/release/core/fabric/mira-nicprobe --json   # RDMA devices, capabilities, matched NIC profile
PYTHONPATH=python python -m miracache.node_prep --hugepage-gib 16               # dry run: hugepages, memlock, RDMA modules
PYTHONPATH=python python -m miracache.node_prep --hugepage-gib 16 --apply --yes # as root, once you agree with the plan
```

Size `--hugepage-gib` to the cache you give each memory node (the example config below uses 16 GiB).

**3. Start three or more memory nodes.** Start from [`deploy/examples/memnode.toml`](https://github.com/mirastacklabs-ai/miracache/blob/main/deploy/examples/memnode.toml)
(addresses, device, hugepage store, certificates) and run them under systemd
([`deploy/systemd/`](https://github.com/mirastacklabs-ai/miracache/blob/main/deploy/systemd/README.md)) or Docker ([`deploy/docker/`](https://github.com/mirastacklabs-ai/miracache/blob/main/deploy/docker/README.md)).
Memory nodes and clients authenticate each other with mutual TLS; use your own CA.

**4. Configure the GPU hosts.** Start from [`deploy/examples/client.toml`](https://github.com/mirastacklabs-ai/miracache/blob/main/deploy/examples/client.toml) and create the
deployment secret once, then copy the same file to every GPU host (never to a memory node). Run as the user the engine
runs as, and keep it on persistent storage: a lost secret makes the whole cache unreadable.

```sh
sudo install -d -m 0700 -o "$USER" /etc/miracache/keys
umask 077; printf 'current = 1\n1 = %s\n' "$(openssl rand -hex 32)" > /etc/miracache/keys/secret
```

**5. Check the engine flags, then launch.** `mira-preflight` hard-fails a host whose engine settings would break
caching or isolation ([`deploy/preflight.md`](https://github.com/mirastacklabs-ai/miracache/blob/main/deploy/preflight.md)):

```sh
PYTHONPATH=python python -m miracache.preflight --engine vllm --engine-args "<your vLLM flags>" \
    --secret-file /etc/miracache/keys/secret
```

<details>
<summary><b>vLLM</b>: the offloading connector with miracache as a secondary tier</summary>

```sh
vllm serve <model> --block-size 16 --prefix-caching-hash-algo sha256_cbor \
  --kv-transfer-config '{"kv_connector":"OffloadingConnector","kv_role":"kv_both",
    "kv_load_failure_policy":"recompute",
    "kv_connector_extra_config":{"spec_name":"TieringOffloadingSpec","cpu_bytes_to_use":34359738368,
      "canonical_layout":true,
      "secondary_tiers":[{"type":"MiraTier","module_path":"miracache.vllm_tier",
        "config":"/etc/miracache/client.toml","kv_load_failure_policy":"recompute"}]}}'
```

`kv_load_failure_policy` must be `recompute` (the default `fail` would turn a cache miss into a request error).
Every host must use the same `PYTHONHASHSEED`, block size and hash algorithm; pipeline and context parallelism must
be 1. If you pass `--revision` or `--quantization`, mirror them in the `MiraTier` entry as `"model_revision"` and
`"ns_fields": {"quantization": "..."}`, so hosts serving different weights never share blocks (preflight checks it).
</details>

<details>
<summary><b>SGLang</b>: HiCache with the dynamic storage backend</summary>

```sh
python -m sglang.launch_server --model-path <model> --page-size 64 \
  --enable-hierarchical-cache \
  --hicache-storage-backend dynamic \
  --hicache-storage-backend-extra-config "$(cat deploy/sglang/extra-config.example.json)"
```

See [`deploy/sglang/README.md`](https://github.com/mirastacklabs-ai/miracache/blob/main/deploy/sglang/README.md) for every required setting.
</details>

**6. Watch it.** Each memory node serves Prometheus metrics on its `[telemetry] listen` address. Import the
dashboards and alert rules from [`deploy/observability/`](https://github.com/mirastacklabs-ai/miracache/blob/main/deploy/observability/README.md).