Local AI

Posted by Wildwood32

Any help getting this setup as a local only installation. Recentlly setup Sparkbox on a Mac Mini 2012 500gb ssd/ 16gb ram. Tried running ollama on it bare with Ubuntu, agonizing slow. Trying sparkbox?

3 replies

Chris wrote:

Yes—SparkBox includes a Local AI option that runs Ollama with a chat interface on your own server; it does not send chats to a cloud AI service. It still uses the Mini’s CPU/RAM, though, so it won’t automatically make a model run faster, and the model size matters a lot on that 2012 hardware. Setup guide: https://tomsparkbox.com/guides/localai-setup.html What exact Ollama model did you try, and is SparkBox installed on Ubuntu on that same Mac Mini?

Wildwood32 wrote:

U havent tried any models yet, still figuring out the setup.

Chris wrote:

That makes sense—you haven’t tried a model yet, so there’s nothing to diagnose as slow. SparkBox’s Local AI runs Ollama and its chat page on your own machine; the Mac’s 16 GB RAM is enough for a small model, though its 2012 CPU may make replies take a while. If SparkBox is installed on Ubuntu on that Mac, your next step is to enable Local AI from the dashboard’s Apps page; the setup guide is here: https://tomsparkbox.com/guides/localai-setup.html. If it isn’t installed there yet, tell me where SparkBox is running and I’ll point you to the right next step.