latest stable 1.8.0 · Apache-2.0

Own YourInference

All local. All yours.

hal0 runs several specialised models side by side on your own hardware, each in a named slot. Your apps and agents talk to the slot, so you swap the model behind it from a dropdown and your apps keep the name they already use.

Read the docs★ Star on GitHubLinux + systemd · one-line install
hal0-api :8080 ready·slots 4 / 10 up·probe strix-halo
host halo·116 GB unified · iGPU + XDNAinteractive demo
agent:8081
rocmagent
mem
16.4 GB
ctx
64k
state
serving
serving · 1 request in flight
coder:8082
vulkancoding
mem
8.9 GB
ctx
—
state
ready
ready · warm, idle
embed:8083
Qwen3 Embedding 0.6B (Q8_0)fits · ~1 gb
vulkanembedding
mem
0.6 GB
ctx
4k
state
ready
feeds Open WebUI document search
img:8188
SDXL Turbo · ComfyUIfits · ~7 gb
rocmcomfyui
mem
6.9 GB
ctx
—
state
ready
POST /v1/images/generations
connected clients✓ no client reconfigured
Open WebUI
agent, coder
Hermes
hal0/agent
pi
agent
Hindsight
hal0/memory
GET /v1/models →agentcoderbrainutility…
install.sh
Linux x86_64 · Python ≥3.12
$curl -fsSL https://hal0.dev/install.sh | sudo bash
/ the idea

One giant model is the wrong shape for a home box.

A 128 GB box has room for several specialists. hal0 gives each job its own slot with its own model and tune, behind one OpenAI-compatible API. Clients name the slot; hal0 keeps the model, the flags and the placement straight.

slot = placement

A name, a device, a port

One model in its own container behind the API, with a type (chat, embedding, rerank, speech, image), a device (ROCm, Vulkan, NPU, CPU), a priority and a name your apps use. Custom slots are one command.

agent · coder · brain · embed · rerank · stt · tts · img · custom: research
profile = tune

The whole tune, reusable

Context length, thinking, vision, speculation, runner and flags live in a named profile. Clone a seed, change it, point any slot at it. The slot drawer shows the resolved command with every flag labelled by owner.

chat · agent · coding · embedding · brain · flm · comfyui
model = facts

The file and nothing else

What is true about the file: size, quantisation, context ceiling, what it can do. Pull from Hugging Face with a fit check, keep several quants, and let update checks tell you when the repo changed.

GGUF · drafters for speculation · HF update checks

One API, one slot per medium. hal0 routes by endpoint and model name, and it never reads a prompt to choose a slot.

/v1/chat/completionsagent, coder, brain
/v1/embeddings · /v1/rerankembed, rerank
/v1/audio/transcriptionsstt (NPU or CPU)
/v1/audio/speechtts
/v1/images/generationsimg (ComfyUI)

Slots run in parallel as far as the memory pool allows; hal0 checks fit before loading and evicts by priority under pressure. Image generation borrows the GPU while it renders, so GPU chat slots pause until it finishes.

/ slots as models

Swap the model, keep the name.

Open WebUI, Hermes, pi and the memory engine are pointed at slot names, not model files. When a better model lands, pick it from the slot's dropdown. The slot reloads and every chat client keeps the config it already has.

  • Open WebUImodel picker lists every chat slot as agent · Qwen3.8 27B, updated within seconds of a change
  • Hermesdefaults to hal0/agent and discovers every other slot
  • pidefaults to agent; sees new slots after a restart
  • Hindsightextracts memory through hal0/memory, which resolves to the slot you choose
  • your codeany OpenAI client with model: "coder" and base_url: http://halo:8080/v1
GET /v1/models returns one entry per chat slot, named after the slot.
model: "agent" is rewritten to whatever the slot holds right now.
Swap from the card, the CLI or MCP. A fit check runs first; a bad id leaves the slot untouched. Requests during the reload get a 503 with Retry-After.
Slot card on the hal0 console with the model picker open, showing models with fit chips
Slot card, picker open. Each option shows size, fit and who else uses it.
/ privacy

Your prompts stay on your box.

Chats, files, embeddings, voice, and the million-dollar idea you brainstormed at 2 a.m. are processed on hardware you own, and they stay there unless you connect an upstream. hal0 sends no telemetry and needs no account.

Local inference

Every model runs in a container on your box. Chat, embedding and speech slots sit on loopback ports behind the API. Realtime voice chains local speech-to-text, chat and text-to-speech with no cloud hop.

No telemetry

hal0's own code has no telemetry. Crash reporting does not ship: there is nothing to switch on and no key to supply. Benchmark bundles and diagnostics never upload on their own.

No account

No sign-up and no licence server. Apache-2.0, with the source on GitHub.

Updates you trigger

Cosign-verified releases with one-flag rollback. While the console is open it checks for new releases and model updates; those checks carry no prompts or files.

on your LAN The API, Open WebUI and the ComfyUI editor serve your network with no login and no TLS by default. Keep the box on your LAN, or turn on key auth and put a reverse proxy in front before you share it. Key auth covers the API only.
/ wired on first boot

The apps come configured.

One install brings up the API, a chat UI, an agent and memory, already pointed at each other, and pulls a small model for the steward. You pick the models for the other slots from the console. Add a slot and it appears in Open WebUI and Hermes within seconds; pi sees it after a restart.

chat

Open WebUI

Installed and started on :3001, with every chat slot in its model picker. Voice is wired from the start; document search on hal0 embeddings and the ComfyUI image button switch on once embed and img have models.

agent

Hermes

A third-party agent, pinned to a reviewed release and installed by default. Defaults to the agent slot and gets hal0's admin and memory tools. Risky tool calls wait for your approval.

coding

pi

A third-party terminal coding agent, one command to add. Points at the agent slot with the same admin and memory tools.

images

ComfyUI

Runs in the img slot. POST /v1/images/generations renders SDXL, SD 1.5 or Qwen-Image. The editor stays on the LAN for workflows the API does not cover.

Bring your own MCP client: /mcp/admin and /mcp/memory are plain Streamable HTTP. Claude Code, Codex and Cursor connect by pasting one line. Destructive admin calls wait until a person says yes.

Hermes, the bundled hal0 agent
Hermes
qwen3.8-27b
ctx0/64K
remote control · self-improving · orchestration
READYtap · abilities
Hermes · abilities
Ghost Relay40pwr
Connect Telegram or Discord through its gateway.
Engram60pwr
Reads and writes the shared memory bank over MCP.
Skills
voice · ttsspeech · sttimage-genvisionembeddings
logspersona
Run agents →
/ memory

One memory your agents share.

A Hindsight engine runs on loopback with local embeddings. Hermes and pi are wired to it automatically, with shared and private banks. Anything that speaks MCP can connect to it.

  • Directives are standing instructions an agent sees every turn.
  • Mental models are standing questions whose answers stay fresh as facts change.
  • The bank workspace shows every fact, a graph of how they connect, and lets you curate or delete with an audit row.
hal0 memory bank workspace with a facts list and graph
Memory bank. Facts, the graph, directives and mental models in one workspace.
/ the console

Every slot, model and agent in one console.

Slots with their pickers and live metrics, a dock with the GPU's state and a merged event feed, profiles, stacks, image generation, memory, MCP, logs and settings. Dark by default, works on a phone.

hal0 console slots page with slot cards, throughput and the memory ruler
Slots. Each card carries a model picker, a profile picker, context and live memory.
Slot drawer open over the slots page: model, profile and the tune block for the agent slot
Slot drawer. Model, profile and the tune: context, thinking, effort, budget, vision and speculation, each marked profile or override. Per-slot tune is on nightly today.
Model drawer open over the slots page: file facts, runner compatibility and source paths for a GGUF
Model drawer. The file and nothing else: quant, size, architecture, drafter, chat template, native context, where it runs.
hal0 overview page with the memory pool, the slots serving now, agents and delegated runs
Overview (preview). What each slot is serving, the memory pool, agents and delegated runs on one page. Nightly channel today.
Image generation pane on the hal0 console with a prompt, size presets, a fit check and a finished render
Image gen. Prompt, size, steps and a fit check before the render; results land beside it.
Agents page with cards for Hermes, pi and a runtime on the roadmap
Agents. Every agent in the runtime as a card: version, model, persona, approvals and health.
The Hermes agent card flipped to its console with restart, logs, persona and approval controls
Card console. Flip a card for restart, logs, tool activity, persona and command approvals.
MCP page listing the bundled hal0 servers with their tool counts and the bridge clients that call them
MCP. The bundled servers with their tool counts, and the bridge clients that call them. The bridge is on nightly today.
Memory bank in its web view: facts as a force-directed graph linked by meaning, time and shared entity, with a tag map beside it
Memory graph. A bank as a web: related meaning, near in time, shared entity, with the tag map beside it.
The dock with its GPU tab open
Dock. GPU state, slot and service health, downloads and a merged event feed.
/ hardware

Strix Halo first. Linux everywhere.

Built and benchmarked on the Ryzen AI Max+ 395 with its 128 GB unified pool, XDNA NPU and RDNA 3.5 iGPU. The same install runs on other Linux boxes, with less headroom.

reference
Ryzen AI Max+ 395
128 GB unified · iGPU + NPU · all published numbers
first-class
Ryzen AI Max 385 / 390
64 GB unified · same install path
supported
Radeon RX 7000
16 to 24 GB · ROCm or Vulkan
experimental
NVIDIA RTX
opt-in CUDA runner · not in the release gate
fallback
CPU only
small models, speech, smoke tests
supported
Proxmox LXC
one script · GPU + NPU passthrough

Linux x86_64 with systemd and podman. No macOS, no Windows, no Docker.

/ alongside the alternatives

Where hal0 sits.

hal0 is the orchestration around llama.cpp, FastFlowLM and ComfyUI, not a new engine, so the table compares the layer above the engine. Several of these tools can be configured otherwise; the cells describe their defaults.

hal0OllamaLM Studiollama-swapLemonade
Clients addressa typed slot name; model swaps behind ita model namean identifier set at loadaliases in hand-edited YAMLa model name
Models resident by defaultseveral, as memory fits, priority evictionup to 3 per GPUone on demand, auto-evictone; groups opt inone per type
Reusable tune profilesnamed, forkable, device-agnosticper-model Modelfileprompt + sampler presetsper-model commandper-model settings
Admin plane as MCPyes, with human approval for risky callsnone foundnone found; hosts MCP toolsnonone found
Memory bundledHindsight, shared and private banksnone foundnone found; document chatnonone found
Agent bundledHermes, plus pi opt-in, with approvalsno; wires external CLIsBionic (closed)nono; GAIA is a separate project
Voice + images, one APISTT, TTS, images, realtimenone foundtranscription via Bionicoptional unified containeryes
Strix Halo + NPUROCm or Vulkan, NPU via FastFlowLMROCm listed; NPU not foundROCm or Vulkan; NPU not foundbring your own server; no NPUROCm + NPU
Licence · telemetryApache-2.0 · none; console checks updatesMIT · none locallyclosed · update checksMIT · none foundApache-2.0 · none

From each project's published documentation as of 2026-11-01. "None found" means we looked and did not find it. Lemonade is also AMD-first; the difference is the slot and profile layer.

first-party sweep

what this box does

all 26 models · sweep 2026-06-19 →
169.8qwen3.5-0.8b
fastest decode · tok/s
26
models measured
6248
peak prefill · tok/s

Sweep from 2026-06-19 on hal0 0.5 with the earlier runner.

Latest release

all releases →
stable1.8.02026-11-20
  • Memory v3: one Memory area, two banks and a guarded embedder switch. The Hindsight memory engine moves to 1.2.0; the dashboard has one top-level Memory area; a new install keeps memory in hal0:shared and hal0:private, and hal0 memory migrate fold-in…
  • An NPU tab and the optional xdna NPU runner. Slots → NPU shows the FLM and GGML lanes with live activity, and xdna runs plain GGUF models on the AMD XDNA2 NPU beside FLM (ADR-0057, ADR-0059).
  • The agent bridge. hal0 bridge add <name> admits a remote coding agent (Claude Code, Codex, a CI job) with a revocable hal0bk_ key of its own, and its hal0-agents, hal0-memory and…
/ questions

Straight answers.

The things people ask before they run the installer.

What is hal0?

An open-source, self-hosted AI inference platform for Linux. It runs chat, embedding, rerank, speech and image models in named slots behind one OpenAI-compatible API, with a web console, a chat UI, agents, MCP servers and a memory engine. Apache-2.0.

Do I need a Strix Halo machine?

No. hal0 is built and benchmarked on the Ryzen AI Max+ 395 first, and it runs on other Linux boxes with an AMD GPU, an NVIDIA GPU (experimental) or only a CPU. Linux x86_64 with systemd and podman is the hard requirement.

Is hal0 an Ollama alternative?

Partly. Both serve local models over an OpenAI-style API. hal0 adds slots that clients address by name, reusable profiles, a console, agents, MCP servers and memory. If you want one binary, or you run macOS or Windows, Ollama is the simpler choice.

Can I run several models at once?

Yes. Each slot is its own container, and how many stay resident depends on your memory pool. hal0 checks fit before loading and evicts by priority under pressure. Image generation borrows the GPU while it renders, pausing GPU chat slots.

Does my data leave the machine?

Inference, chat history, uploaded documents and memory stay on the box unless you connect an upstream. No telemetry, no account. hal0 pulls models and runner images when you ask and checks for updates while the console is open.

Is the API OpenAI-compatible?

Yes: /v1/chat/completions, /v1/embeddings, /v1/rerank, /v1/audio/speech, /v1/audio/transcriptions, /v1/images/generations and a realtime WebSocket. Any OpenAI client works; use a slot name as the model.

Which GPUs and NPUs work?

AMD Strix Halo iGPU on ROCm or Vulkan, plus the XDNA NPU through FastFlowLM and, on the nightlies, through ggml-xdna, which runs any GGUF on the NPU. AMD Radeon RX 7000 on the same runner. NVIDIA through an opt-in CUDA runner, experimental. CPU-only for small models.

forum.hal0.dev

what users are talking about

open the forum ↗
community profiles

optimized model & hardware recipes from the community

all profiles →
learn

latest from the blog

all posts →
/ get hal0

Put the whole stack
on your own box.

One command on a fresh Linux box brings up the API, the chat UI, the agent and memory. Then pick models for your slots from the console.

install.sh
Linux x86_64 · Python ≥3.12
$curl -fsSL https://hal0.dev/install.sh | sudo bash
Apache-2.0Linux + systemdno telemetrysigned releases