Guides 53 pages
On this page

Guides

Run agents on local models

Run agents on models served by your own computer (Ollama, LM Studio, llama.cpp, vLLM, SGLang or any OpenAI-compatible server), add a server that needs a key, and fix the common problems.

Agents can run on models served by a machine of your own. Their requests go to the model server on that machine and never to a cloud model provider, and there are no usage limits beyond what the machine can do.

Cyborg supports Ollama, LM Studio, llama.cpp (llama-server), vLLM and SGLang, plus any other server that speaks the OpenAI API.

Before you start

  • Cyborg must be running on the machine that runs the model server. Agents on a local model live on that machine.
  • Local models through OpenCode must be turned on for that machine (they are off by default). See Turn on local models.
  • OpenCode must be installed on that machine. Local models run through OpenCode, and Cyborg writes its settings for each agent, so you never edit an OpenCode config file. If it is missing, the app offers Install OpenCode (it takes about a minute). You can also run cyborg provider:install <workspace-id> opencode --daemon <machine-id>.
  • Small local models may not use Cyborg tools reliably. Pick a model that supports tool calls.

Turn on local models

New agents can pick a local model only on a machine whose config.json (in ~/.cyborg7 by default) turns the feature on:

"features": {
  "localOpenCode": true
}

Add localOpenCode to the features block if there is one, then restart Cyborg on that machine with cyborg daemon restart. cyborg daemon reload does not apply it; it lists features.localOpenCode among the settings that need a restart.

While it is off, the model picker offers no Local · machine name row, but the machine still finds its model servers: they show in Home → the machine → Local models and in cyborg machine:local-servers, and agents already on a local model keep running.

Going back to an older Cyborg on that machine: remove localOpenCode from features (and the localModels block) first. An older Cyborg checks features strictly and refuses to start with a config.json that has a key it does not know.

Start a server Cyborg can find

Cyborg checks these ports on the machine’s own loopback address (127.0.0.1) about every 30 seconds (a port with nothing on it, every few minutes):

PortServer
11434Ollama
1234LM Studio
8080llama.cpp
8000-8010vLLM and SGLang

Start yours on one of them, for example:

ollama serve
lms server start
llama-server -m ./model.gguf --port 8080 --jinja
vllm serve Qwen/Qwen3-8B --port 8000 --enable-auto-tool-choice --tool-call-parser hermes
python -m sglang.launch_server --model-path Qwen/Qwen3-8B --port 8001 --tool-call-parser qwen25

Agents call tools, so turn tool calling on where the server needs it: --jinja for llama.cpp, --enable-auto-tool-choice with the parser that matches your model for vLLM, and --tool-call-parser for SGLang. SGLang’s own default port (30000) is not checked; use a port from 8000 to 8010 or add the server as described below.

Only servers on the machine’s loopback address are found on their own. A server on another port, on your local network, or one that needs an API key has to be added.

For Ollama, set OLLAMA_CONTEXT_LENGTH (for example to 32768) before ollama serve. Ollama otherwise gives a model a short context, and long agent prompts may be cut.

Add a server that needs a key or uses another port

In the app: Home → the machine → Local models → + Add a server. Only the machine’s owner, or someone with admin access to it, can add servers.

The form has three fields:

  • Name (optional): what the app shows for the server, for example Spark thinker.
  • Address: the server’s address on that machine, for example http://127.0.0.1:8001. /v1 is optional.
  • API key file: a file on that machine that holds the key, for example ~/llm/.api_key. Cyborg reads it there and never copies it. Leave it empty if the server has no key.

Test and add checks the server before anything is saved. When it answers, you see Added vLLM :8001 · 1 model. and the server is used straight away, with no restart.

From a terminal, the same thing:

cyborg machine:local-server-add <workspace-id> <machine-id> http://127.0.0.1:8001 --name "Spark thinker" --key-file '~/llm/.api_key'
cyborg machine:local-servers <workspace-id> <machine-id>

The key file is read on the machine you add the server to, so ~ there is that machine’s home. Quote ~ (as above) so your own shell leaves it alone; unquoted, your shell turns it into your home directory before Cyborg sees it.

Or edit the machine’s config.json (in ~/.cyborg7 by default), then restart Cyborg there with cyborg daemon restart:

"localModels": {
  "endpoints": [
    {
      "id": "spark-thinker",
      "label": "Spark thinker",
      "baseUrl": "http://127.0.0.1:8001",
      "apiKeyFile": "~/llm/.api_key"
    }
  ]
}

cyborg daemon reload does not apply a localModels change; it lists the changed localModels.* settings (for example localModels.endpoints) among those that need a restart. The fields are described in Configuration.

Why only a key file: the key itself never enters config.json, the relay or the agent’s settings. Cyborg reads the file on the machine when it checks the server and passes OpenCode a reference to the file, not the key. Keep the file readable only by you:

chmod 600 ~/llm/.api_key

The file must be a regular file (a link to one is fine) of at most 4 KB, holding the key on one line.

Create an agent on a local model

  1. Click New agent and go to the Model step.
  2. Pick Local · machine name. Its subtitle names the engine and the number of models, for example vLLM · 2 models.
  3. Pick a model. Models are grouped by server (vLLM :8000, vLLM :8001), each with its context size, for example 256K context.
  4. Continue as with any agent. The step reminds you that the agent Runs through OpenCode on <machine>.

Only servers that are running show models here. A server that is down or needs a key is summed up in one line under the list. When no server runs, the row says Start a model server on <machine>: Ollama, LM Studio, llama.cpp, vLLM or SGLang. If yours needs an API key, add it in <machine>'s Local models.

No Local · machine name row at all: local models are not turned on for that machine (see Turn on local models).

See what’s running

Open Home → the machine → Local models. Each server shows its engine, status (Running, Starting, Needs key, Key rejected, Key file unreadable, Not responding), Key set or Needs key, and Detected or Added by you. Each model shows its context size. Check again looks for servers right away.

Load and unload apply to Ollama only. Other servers keep their models loaded while they run.

From a terminal:

cyborg machine:models <workspace-id>
cyborg machine:local-servers <workspace-id> <machine-id>
cyborg machine:model <workspace-id> <machine-id> <model-id> load

To remove a server you added (detected servers come back on their own while they run): Remove on its row, or cyborg machine:local-server-remove <workspace-id> <machine-id> <server-id> --yes. Agents using its models stop working.

Move an existing agent to a local model

Open the agent’s editor, go to Provider, pick Local · machine name and a model, and save. The agent’s next run starts a fresh session on the new model. Agents created earlier on Ollama keep working as they are until you change them this way.

What leaves your computer

So the app can list them, Cyborg tells the relay, for each server on a machine: its id and name (the one you gave it, for example Spark thinker), type, port, version, status, whether a key is set, whether it can load and unload models, when it last answered, and whether its model list was cut short; and for each model, its name, size and context length. That list never carries a server’s address, the key, or the key file’s path. When you add a server from the app or the CLI, its address and the key file’s path (never the key) pass through the relay to that machine. Requests to a model go only to the server on that machine; the agent’s conversation reaches the workspace like any other agent’s. See Security and data protection.

Troubleshooting

No model servers found on <machine>

Cyborg found nothing on the ports above. Start a server on one of them and click Check again, or add it.

Needs key

The server answered 401: it asks for an API key. Add it with its key file (the owner sees Add key file, with the address filled in). Others see Ask <owner> to add its key. An agent on it says <server> on <machine> asks for an API key. Add its key file in <machine>'s Local models.

Key rejected

The key in the file you gave was refused. Check the file on <machine>. The server refused the key in that file. Check the file holds the key the server was started with. After you change the key, restart agents that use the server.

Cyborg can't read that file on <machine>.

The server shows Key file unreadable, and an agent on it says Cyborg can't read <server>'s key file on <machine>. The key file is missing, is not a regular file, is larger than 4 KB, or Cyborg’s user can’t read it, or the path is not absolute: give a full path such as /home/me/llm/.api_key or ~/....

Not responding

The server was seen before, or was added, and no longer answers. It is stopped, still starting, or another program took its port. Start it, then click Check again. An agent using it says <agent> can't reach vLLM :8000 on <machine>. Start the server, then send your message again.

That address answered, but not like a model server. Check the port.

Something else listens on that address. Check the port the model server uses.

Nothing answered at that address on <machine>.

Nothing listens there. Start the server, or check the address.

<server> on <machine> runs on its owner's API key. Only the owner can start agents on it.

The server has a key, and its owner did not share it (shared is false, the default). Only the machine’s owner and admins can start agents on it. The owner can set "shared": true on that server in config.json (see Configuration).

<agent>'s model server isn't on <machine>. Pick another model to keep going.

The server the agent was created on was removed, or its id changed. Pick another model in the agent’s editor.

This computer can't run local models yet. Try again in a moment.

Cyborg on that machine is still starting and has not looked for model servers yet. Send the message again in a moment.

<agent>'s model, <model>, is no longer on <server> on <machine>.

The model was renamed or removed from the server. Pick another model in the agent’s editor (Pick a model). Cyborg never switches the agent to another model by itself.

Needs OpenCode

OpenCode is not installed on that machine. The owner can click Install OpenCode; others see Local models need OpenCode on <machine>. Ask <owner> to install it.

vLLM keeps its models loaded while it runs; there's nothing to load or unload.

Load and unload apply to Ollama only.

A model the agent doesn’t see

Cyborg skips a model whose name has characters other than letters, digits and . _ : / @ + -. Rename it on the server (vLLM: --served-model-name).

Ollama gives this model a <n>K context.

Ollama’s context for that model is under 32K. Set OLLAMA_CONTEXT_LENGTH and restart Ollama.

The first reply is slow

OpenCode starts once for each agent, and a large model may need a while for its first answer. Later replies are faster.