Own YourInference
All local. All yours.
hal0 runs several specialised models side by side on your own hardware, each in a named slot. Your apps and agents talk to the slot, so you swap the model behind it from a dropdown and your apps keep the name they already use.
- Qwen3.8 27B (UD Q4_K_XL)fits · ~16 gb16.4 GB · hal0 default · vision · tool calling
- LFM2.5 2.6B (Q8_0)fits · ~3 gb2.7 GB · seeded for brain · tool calling
- Qwen3 30B A3B (Q4_K_M)fits · ~17 gb17.3 GB · Hugging Face · unsloth/Qwen3-30B-A3B-GGUF
- Qwen3.5 9B (Q8_0)fits · ~9 gb8.9 GB · Hugging Face · unsloth/Qwen3.5-9B-GGUF
- Qwen3 30B A3B (Q4_K_M)fits · ~17 gb17.3 GB · Hugging Face · unsloth/Qwen3-30B-A3B-GGUF
- Qwen3.5 0.8B (UD Q4_K_XL)fits · ~1 gb0.5 GB · hal0 catalogue · small and quick
agentcoderbrainutility…One giant model is the wrong shape for a home box.
A 128 GB box has room for several specialists. hal0 gives each job its own slot with its own model and tune, behind one OpenAI-compatible API. Clients name the slot; hal0 keeps the model, the flags and the placement straight.
A name, a device, a port
One model in its own container behind the API, with a type (chat, embedding, rerank, speech, image), a device (ROCm, Vulkan, NPU, CPU), a priority and a name your apps use. Custom slots are one command.
The whole tune, reusable
Context length, thinking, vision, speculation, runner and flags live in a named profile. Clone a seed, change it, point any slot at it. The slot drawer shows the resolved command with every flag labelled by owner.
The file and nothing else
What is true about the file: size, quantisation, context ceiling, what it can do. Pull from Hugging Face with a fit check, keep several quants, and let update checks tell you when the repo changed.
One API, one slot per medium. hal0 routes by endpoint and model name, and it never reads a prompt to choose a slot.
Slots run in parallel as far as the memory pool allows; hal0 checks fit before loading and evicts by priority under pressure. Image generation borrows the GPU while it renders, so GPU chat slots pause until it finishes.
Swap the model, keep the name.
Open WebUI, Hermes, pi and the memory engine are pointed at slot names, not model files. When a better model lands, pick it from the slot's dropdown. The slot reloads and every chat client keeps the config it already has.
- Open WebUImodel picker lists every chat slot as
agent · Qwen3.8 27B, updated within seconds of a change - Hermesdefaults to
hal0/agentand discovers every other slot - pidefaults to
agent; sees new slots after a restart - Hindsightextracts memory through
hal0/memory, which resolves to the slot you choose - your codeany OpenAI client with
model: "coder"andbase_url: http://halo:8080/v1
GET /v1/models returns one entry per chat slot, named after the slot.model: "agent" is rewritten to whatever the slot holds right now.
Your prompts stay on your box.
Chats, files, embeddings, voice, and the million-dollar idea you brainstormed at 2 a.m. are processed on hardware you own, and they stay there unless you connect an upstream. hal0 sends no telemetry and needs no account.
Local inference
Every model runs in a container on your box. Chat, embedding and speech slots sit on loopback ports behind the API. Realtime voice chains local speech-to-text, chat and text-to-speech with no cloud hop.
No telemetry
hal0's own code has no telemetry. Crash reporting does not ship: there is nothing to switch on and no key to supply. Benchmark bundles and diagnostics never upload on their own.
No account
No sign-up and no licence server. Apache-2.0, with the source on GitHub.
Updates you trigger
Cosign-verified releases with one-flag rollback. While the console is open it checks for new releases and model updates; those checks carry no prompts or files.
The apps come configured.
One install brings up the API, a chat UI, an agent and memory, already pointed at each other, and pulls a small model for the steward. You pick the models for the other slots from the console. Add a slot and it appears in Open WebUI and Hermes within seconds; pi sees it after a restart.
Open WebUI
Installed and started on :3001, with every chat slot in its model picker. Voice is wired from the start; document search on hal0 embeddings and the ComfyUI image button switch on once embed and img have models.
Hermes
A third-party agent, pinned to a reviewed release and installed by default. Defaults to the agent slot and gets hal0's admin and memory tools. Risky tool calls wait for your approval.
pi
A third-party terminal coding agent, one command to add. Points at the agent slot with the same admin and memory tools.
ComfyUI
Runs in the img slot. POST /v1/images/generations renders SDXL, SD 1.5 or Qwen-Image. The editor stays on the LAN for workflows the API does not cover.
Bring your own MCP client: /mcp/admin and /mcp/memory are plain Streamable HTTP. Claude Code, Codex and Cursor connect by pasting one line. Destructive admin calls wait until a person says yes.

One memory your agents share.
A Hindsight engine runs on loopback with local embeddings. Hermes and pi are wired to it automatically, with shared and private banks. Anything that speaks MCP can connect to it.
- Directives are standing instructions an agent sees every turn.
- Mental models are standing questions whose answers stay fresh as facts change.
- The bank workspace shows every fact, a graph of how they connect, and lets you curate or delete with an audit row.

Every slot, model and agent in one console.
Slots with their pickers and live metrics, a dock with the GPU's state and a merged event feed, profiles, stacks, image generation, memory, MCP, logs and settings. Dark by default, works on a phone.










Strix Halo first. Linux everywhere.
Built and benchmarked on the Ryzen AI Max+ 395 with its 128 GB unified pool, XDNA NPU and RDNA 3.5 iGPU. The same install runs on other Linux boxes, with less headroom.
Linux x86_64 with systemd and podman. No macOS, no Windows, no Docker.
Where hal0 sits.
hal0 is the orchestration around llama.cpp, FastFlowLM and ComfyUI, not a new engine, so the table compares the layer above the engine. Several of these tools can be configured otherwise; the cells describe their defaults.
| hal0 | Ollama | LM Studio | llama-swap | Lemonade | |
|---|---|---|---|---|---|
| Clients address | a typed slot name; model swaps behind it | a model name | an identifier set at load | aliases in hand-edited YAML | a model name |
| Models resident by default | several, as memory fits, priority eviction | up to 3 per GPU | one on demand, auto-evict | one; groups opt in | one per type |
| Reusable tune profiles | named, forkable, device-agnostic | per-model Modelfile | prompt + sampler presets | per-model command | per-model settings |
| Admin plane as MCP | yes, with human approval for risky calls | none found | none found; hosts MCP tools | no | none found |
| Memory bundled | Hindsight, shared and private banks | none found | none found; document chat | no | none found |
| Agent bundled | Hermes, plus pi opt-in, with approvals | no; wires external CLIs | Bionic (closed) | no | no; GAIA is a separate project |
| Voice + images, one API | STT, TTS, images, realtime | none found | transcription via Bionic | optional unified container | yes |
| Strix Halo + NPU | ROCm or Vulkan, NPU via FastFlowLM | ROCm listed; NPU not found | ROCm or Vulkan; NPU not found | bring your own server; no NPU | ROCm + NPU |
| Licence · telemetry | Apache-2.0 · none; console checks updates | MIT · none locally | closed · update checks | MIT · none found | Apache-2.0 · none |
From each project's published documentation as of 2026-11-01. "None found" means we looked and did not find it. Lemonade is also AMD-first; the difference is the slot and profile layer.
what this box does
Sweep from 2026-06-19 on hal0 0.5 with the earlier runner.
Latest release
all releases →- Memory v3: one Memory area, two banks and a guarded embedder switch. The Hindsight memory engine moves to 1.2.0; the dashboard has one top-level Memory area; a new install keeps memory in hal0:shared and hal0:private, and hal0 memory migrate fold-in…
- An NPU tab and the optional
xdnaNPU runner. Slots → NPU shows the FLM and GGML lanes with live activity, and xdna runs plain GGUF models on the AMD XDNA2 NPU beside FLM (ADR-0057, ADR-0059). - The agent bridge. hal0 bridge add <name> admits a remote coding agent (Claude Code, Codex, a CI job) with a revocable hal0bk_ key of its own, and its hal0-agents, hal0-memory and…
Straight answers.
The things people ask before they run the installer.
What is hal0?
An open-source, self-hosted AI inference platform for Linux. It runs chat, embedding, rerank, speech and image models in named slots behind one OpenAI-compatible API, with a web console, a chat UI, agents, MCP servers and a memory engine. Apache-2.0.
Do I need a Strix Halo machine?
No. hal0 is built and benchmarked on the Ryzen AI Max+ 395 first, and it runs on other Linux boxes with an AMD GPU, an NVIDIA GPU (experimental) or only a CPU. Linux x86_64 with systemd and podman is the hard requirement.
Is hal0 an Ollama alternative?
Partly. Both serve local models over an OpenAI-style API. hal0 adds slots that clients address by name, reusable profiles, a console, agents, MCP servers and memory. If you want one binary, or you run macOS or Windows, Ollama is the simpler choice.
Can I run several models at once?
Yes. Each slot is its own container, and how many stay resident depends on your memory pool. hal0 checks fit before loading and evicts by priority under pressure. Image generation borrows the GPU while it renders, pausing GPU chat slots.
Does my data leave the machine?
Inference, chat history, uploaded documents and memory stay on the box unless you connect an upstream. No telemetry, no account. hal0 pulls models and runner images when you ask and checks for updates while the console is open.
Is the API OpenAI-compatible?
Yes: /v1/chat/completions, /v1/embeddings, /v1/rerank, /v1/audio/speech, /v1/audio/transcriptions, /v1/images/generations and a realtime WebSocket. Any OpenAI client works; use a slot name as the model.
Which GPUs and NPUs work?
AMD Strix Halo iGPU on ROCm or Vulkan, plus the XDNA NPU through FastFlowLM and, on the nightlies, through ggml-xdna, which runs any GGUF on the NPU. AMD Radeon RX 7000 on the same runner. NVIDIA through an opt-in CUDA runner, experimental. CPU-only for small models.
what users are talking about
optimized model & hardware recipes from the community
0.6 GB. Not a chat model — the draft half of a speculative pair, pinned so the big model never waits on it.
draft-mtp with an f16 KV cache. The fastest tokens this box produces, at the cost of 19 GB resident and no room for a second big slot.
Sized for autocomplete rather than long generations: 28.5 GB resident, prefill above 750 tok/s so a whole file lands fast.
latest from the blog
Put the whole stack
on your own box.
One command on a fresh Linux box brings up the API, the chat UI, the agent and memory. Then pick models for your slots from the console.