Wally

Run open models on your own machine, or hosted when the job outgrows it.

Chat, vision, speech and embeddings, all from the terminal. Local models never leave the device; hosted ones go through a console you sign in to and are metered against your own credit.

wally pull qwen3     # download a model
wally run qwen3      # chat with it, offline

That is the whole first run. No account, no key, nothing leaves the machine.

Signing in

Models you have pulled run on this machine and need no account. To use a hosted model instead, sign in to a RunAnywhere console:

wally login          # opens a browser; approve it there
wally whoami         # the account you are signed in as
wally usage          # credit, and what you have used this month
wally logout

The terminal never asks for a password. It shows a code, you approve it in the browser, and it collects an API key with your credit behind it. That key appears on the console's Cloud keys page and can be revoked there at any time.

Against a console running on your own machine:

export WALLY_CONSOLE_URL=http://localhost:8002
wally login

Then hand a hosted model to a coding session:

wally opencode -m gemma-4

For the Open Frontier hosted path, make the choice explicit and pass any OpenCode arguments after --:

wally opencode --cloud --model <console-model-id> -- --agent build

--cloud never falls back to a local model. wally opencode -m <model> runs whichever model you name, local or hosted.

If the model is on this machine, wally serves it locally. If it is not, the request goes to the console you are signed in to, is checked against your balance before it runs, and is metered.

Install

macOS (Apple Silicon)

brew install runanywhereai/wally/wally

or

curl -fsSL https://raw.githubusercontent.com/RunanywhereAI/wally/main/install.sh | sh

From source

Needs a built SDK kit, not SDK source:

cmake -B build -DWALLY_SDK_KIT=<sdks>/dist/cpp-desktop-macos-arm64
export WALLY_SDK_SWIFT_PATH=<sdks>          # for the MLX backend on Apple
cmake --build build -j8

build/wally is the full binary. build/wally-cxx is the same CLI without MLX, and is what you get if WALLY_SDK_SWIFT_PATH is unset.

MLX loads its Metal shaders from mlx-swift_Cmlx.bundle next to the executable, so install the pair together:

mkdir -p ~/.local/lib/wally
cp -R build/mlx-swift_Cmlx.bundle build/wally ~/.local/lib/wally/
printf '#!/bin/sh\nexec "$HOME/.local/lib/wally/wally" "$@"\n' > ~/.local/bin/wally
chmod +x ~/.local/bin/wally

Copy the binary on its own and MLX will not register.

Windows (x64)

irm https://raw.githubusercontent.com/RunanywhereAI/wally/main/install.ps1 | iex

Linux (x86_64)

No Linux release asset is currently published. Use the source build below; install.sh intentionally fails instead of claiming that an unavailable bottle was installed.

Get started

wally pull qwen3          # download
wally run qwen3           # chat
wally run qwen3 "Hello"   # one-shot
wally serve qwen3         # OpenAI-compatible API on :8080 (macOS/Linux)

wally models list --all is the full catalog. Short names work everywhere (qwen3, llama3.2, whisper-tiny, piper, …). Any Hugging Face GGUF works too:

wally pull hf.co/Qwen/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf

Backends

One wally binary. Catalog models already name their engine (GGUF → llama.cpp, mlx-* → MLX, Core ML → NeuRT, QNN-context → QHexRT). You normally do not pick one.

Override only when you mean it:

wally llm generate --engine mlx -m mlx-qwen3 "Hello"
wally run --engine qhexrt /path/to/lfm2_5_230m_HNPU "Hello"
wally image generate --engine neurt --prompt "a red cube" --out out.png

--engine accepts mlx, llamacpp, sherpa, onnx, neurt / coreml / ane, and qhexrt / qnn / npu / hexagon. If you omit it, commons picks the highest-priority registered backend that implements that primitive:

wally backends is the source of truth for this binary. Public bottles never list neurt or qhexrt. Those engines are private overlays, never Homebrew / GitHub Release assets.

Where each engine exists

Public Windows ARM64 kits are commons-only (no llama.cpp / ONNX / Sherpa on MSVC ARM64). Snapdragon NPU is overlay-only. x64 Windows has no Hexagon path.

Modalities × engines

Yes = this engine implements the primitive. Try = a catalog id that wally pull / a local path can run. Overlay engines still need the matching on-disk bundle (compiled .mlmodelc tree, or *_HNPU / v81/ QNN-context dir) — a Hugging Face repo page is HTML, not a model.

MLX registers with a one-line -811 then Swift callbacks install it — that warning is expected. image generate is compiled only when NeuRT is linked; --prompt and --out are required (not a positional prompt). --steps 4 is enough for a smoke PNG.

QHexRT on device also needs QAIRT matching the Hexagon skel (QNN_SDK_ROOT + ADSP_LIBRARY_PATH=…\lib\hexagon-v81\unsigned on v81). Overlay 2.47 DLLs vs a 2.41/2.48 device skel will fail to instantiate graphs. Pass the *_HNPU directory, not a GGUF. GGUF files cannot run on the ARM64 overlay binary (no llama.cpp).

Models

Catalog models are grouped by the org that trains them. GGUF rows run on llama.cpp (macOS, Windows x64, Linux). mlx-* rows run on Apple Silicon only.

Language

Vision

wally vlm generate --model smolvlm2 --image photo.png "What is in this picture?"

Speech