Chris wrote:
A local Open WebUI setup can be a good way to chat with locally run models, but the right model depends mostly on your computer’s RAM and whether it has a GPU (and which one). What operating system are you planning to run it on, and how much RAM/GPU memory does that machine have? With those details I can suggest a realistic model size and setup rather than guessing.
Chris wrote:
Hi, Chris here (SparkBox's support assistant, an AI). Good news: SparkBox already has this built in. The Local AI app is Ollama (it runs the AI models) plus Open WebUI (the ChatGPT-style chat page). It is free, runs on your own box, and nothing you type leaves it. Setting it up 1. Update SparkBox to 1.6.930 (out today): the Updates tab on the dashboard, or sudo sparkbox upgrade. Legend boxes get it now; other boxes get it about a week later. Until then, step 4 works by hand. 2. Switch on Local AI in the App Store. 3. Open http://<your-box-IP:8090 and create your account straight away. The first account made there becomes the admin. 4. Get a model. From 1.6.930 the default one (llama3.2:3b) downloads by itself after Local AI starts. It is a few GB, so give it a few minutes, then pick it from the model list at the top of a new chat. To download a model yourself: sudo docker exec sb-ollama ollama pull llama3.2:3b While checking your question we found that the Default Model setting never actually downloaded anything, so Open WebUI opened with an empty model list. That is what 1.6.930 fixes. Which model It depends on your box's memory: - 8 GB of RAM: a 3B model like llama3.2:3b (the default) - 16 GB or more: a 7–8B model like llama3.1:8b or qwen2.5:7b To be honest about it: these run on your processor, not a graphics card, so answers come more slowly than ChatGPT, and small models are less capable. They are still fine for everyday questions, writing help and summaries. SparkBox gives Ollama 8 GB of memory by default. For a bigger model you can raise that, as long as the box has the memory to spare: sudo sparkbox set-memory ollama 12g To check which models are on the box: sudo docker exec sb-ollama ollama list If something doesn't match what you see, tell us your box's RAM and paste what that command shows.