AI NPC docs
Get it on Nexus

Guides

Run the AI on another PC

Let an old PC with a graphics card write the replies while your gaming PC keeps all its power for the game.

An old PC with a graphics card can write the replies while your gaming PC keeps all its power for the game. The two PCs talk over a network cable (or your router), and the mod asks the second PC for every reply through the Custom provider. This guide shows one way only: llama.cpp's own server, with the same small model as Local AI. It runs on any graphics card with Vulkan, a GTX 1060 6 GB included, and you download two files.

What you need. A second Windows PC with a graphics card (a GTX 1060 6 GB is plenty), 8 GB of RAM, 2 GB of free disk space, and a network cable between the two PCs, or both PCs on the same router. If Ollama runs on that PC, quit it first (right-click its tray icon, Quit), so it does not hold the card.

On the second PC

  1. Download llama.cpp's Windows Vulkan build from github.com/ggml-org/llama.cpp/releases. Open the newest release and take the file named llama-b<number>-bin-win-vulkan-x64.zip (about 37 MB). Vulkan is chosen on purpose: it needs no CUDA files and works on NVIDIA, AMD and Intel cards.

  2. Download the model Ternary-Bonsai-2-4B-Q2_0.gguf (1.2 GB) from the Files tab of huggingface.co/prism-ml/Ternary-Bonsai-2-4B-gguf. It is the model inside the Local AI pack. If that pack is installed on your gaming PC, the same file is in red4ext\plugins\AiNpcCyberpunkMod\app\local-ai\models of the game folder, and a USB stick carries it over.

  3. Make a folder C:\ai. Extract the zip into C:\ai\llama, so that llama-server.exe sits directly inside it. Put the .gguf file in C:\ai.

  4. Open a Command Prompt in C:\ai (type cmd in the folder's address bar and press Enter) and run:

    llama\llama-server.exe -m Ternary-Bonsai-2-4B-Q2_0.gguf --host 0.0.0.0 --port 8080 -ngl 99 -c 8192 -np 1 -fa on --reasoning off --alias bonsai

    --host 0.0.0.0 lets other PCs reach the server, -ngl 99 puts the whole model on the graphics card, and --alias bonsai is the model name you enter in the mod later. It is ready when it says listening on http://0.0.0.0:8080. Leave the window open. To stop it, press Ctrl+C or close the window, and to start it again run the same command (the Up arrow brings it back). No restart of the PC is ever needed.

  5. Windows asks whether to let the program through the firewall. Tick both Private networks and Public networks, then press Allow access. A direct cable counts as a public network, so the second tick matters. If you pressed Cancel or closed the box, Windows blocks the program for good: open a Command Prompt as administrator, run netsh advfirewall firewall delete rule name="llama-server.exe", start the server again and answer the question.

Connect the two PCs

  1. Plug the cable into both PCs. With no router, Windows gives each PC an address starting with 169.254 by itself, which can take up to a minute. Both PCs may call the network "Unidentified network": that is normal.
  2. On the second PC, open a Command Prompt and run ipconfig. In the block of the wired adapter ("Ethernet adapter Ethernet"), the "Autoconfiguration IPv4 Address" (or "IPv4 Address") is the address, for example 169.254.37.12. Through a router it starts with 192.168 and works the same.
  3. On the gaming PC, open a browser and go to http://169.254.37.12:8080/v1/models, with your address. Some text starting with {"models": means the two PCs are talking. If the page never loads, the firewall (step 5), the cable or the address is the cause.

In the mod, on the gaming PC

  1. Open AI provider, press Open on the Custom endpoint row, copy http://169.254.37.12:8080/v1 (with your address) and press Paste endpoint URL. Copy bonsai and press Paste model ID. No key is needed. Press Test connection: it should say OK.
  2. Go back, choose Custom (OpenAI-compatible) as the provider and apply. Replies now come from the second PC, and the voices still run on the gaming PC.

Is the second PC's graphics card doing the work? Open Task Manager on it, Performance, then the NVIDIA GPU. When the server starts, "Dedicated GPU memory" rises by about 1.5 to 2 GB, and the 3D or Compute graph moves while a reply is written. If the memory does not rise and the processor is busy, run llama\llama-server.exe --list-devices. Your card must be in the list, for example Vulkan0: NVIDIA GeForce GTX 1060. If it is missing, update the graphics driver (Vulkan comes with it). If an Intel chip is listed beside your card, add --device Vulkan1 (the number of your card) to the command.

Speed. On my RTX 4080 Laptop this exact setup writes about 140 tokens a second and reads a 7,000 token prompt in 1.7 s, so the first reply with a character starts in under a second and later ones in about 0.1 s. A GTX 1060 is slower: my estimate, not a measurement, is a third to a half of the writing speed and several times slower on the long first prompt. Expect a few seconds for the first reply with a character and one to two seconds after that, instead of minutes. Two minutes per reply with the card idle means the model ran on the processor, which the steps above avoid.