Hey Tom and others that might weigh in as well. I have just had an Immich issue importing a lot of files (12 years worth of pictures and videos/1.4TB). I kept crashlooping when trying to index, what TomAI says are too much. With a lot of help from TomAI we finally got Immich to just start up (couldn't even enter Immich, because of the loops), and changed the jobsettings to 1 for each job, so my NAS could actually handle it. Now it runs. Just slowly. Maybe there need to be an easier sparkbox fix, for stopping such a loop, or maybe I just don't know what I'm doing, and putting too much pressure on it, because I didn't know better. Thought 8GB RAM was plenty for this. TomAI says differently, and tells me my NAS is actively "minimizing" RAM usage just to keep up. Let me say this real quick. I actually bought a Synology first with 2GB RAM, and which supportet 8GB at most (2x 4GB RAM). I feel like I dodged a bullet, when I returned it unopened and bought the Ugreen DXP 4800 Pro instead, for just a little more. Well TomAI says the permanent fix would be to upgrade my RAM, so I won't have any issues going forward running both Immich, Medieserver and all the other good things on Sparkbox. I read Tom's article where he says 4GB RAM could go, but he would recommend 8GB for running Sparkbox. Now TomAI says 32GB RAM should do it, and that 64GB RAM would be nice too if I planned on running more in the future, or just to futureproof it. I would like something that just runs, but also not spend more than actually neccesary. This project (getting a NAS, backing up my photos, documents and getting a personal medieserver) have already cost me a lot more than first anticipated (but also opened my eyes to a world I didn't even imagine was possible). For reference, where I'm living the cheapest "Crucial DDR5 RAM 32GB Kit (2x16GB) 5600MHz SODIMM Memory for Laptop/Notebook, Mini PC (Also 5200MHz / 4800MHz) CL46 - CT2K16G56C46S5" I can find is 335EUR for 2x16Gb or 700EUR for 2x32GB, which is almost what I paid for my NAS in the first place excluding Harddrives of course. I have also heard adding one of two SSDΒ΄s can help a lot with processing speed, but don't completely understand it. Would like to know Tom and your guys view and opinion on what would/could be my next move. I currently have this installed out of the box "CPU 13th Gen Intel(R) Core(TM) i3-1315U 4500 6 Cores 8 Threads 54 Β°C Memory (Samsung) M425R1GB4BB0-CWMOD 8 GB 5600 MHz". TomAI suggested Crusial or Kingston as proven quality brands. So do I spend the money and buy 2x16GB or 2x32GB, One or two SSDΒ΄s or maybe nothing at all or just a single 8GB RAM identical to my current RAM card (samsung)? Would love that my machine was turning down jobs just to follow aling, but just works. Thanks for your help educating a noob to this π
36 replies
tomspark wrote:
You're not doing anything wrong, a 1.4TB / 12-year library is genuinely heavy, and Immich's photo-analysis is about the most demanding thing you can run. The crash-looping on a big first import is a rough edge I've already flagged to smooth out, so don't take it as your fault. On the spend: go 32GB (the 2x16GB kit, ~335EUR) and stop there. That's the comfortable amount for Immich plus a media server on your i3; 64GB is real money for headroom you almost certainly won't use. Swap both sticks for a matched pair rather than adding a lone 8GB. The bigger speed win is an SSD: put SparkBox's apps and database on an M.2 NVMe drive (your DXP4800 has the slots) and keep your hard drives for the actual photos and videos. Immich's indexing is very database-heavy, so that one change speeds it up a lot. One NVMe is plenty; a second only if you want a mirror for safety. So: 32GB RAM + one NVMe SSD is the sweet spot, skip the 64GB. Want me to walk you through pointing SparkBox at the SSD once it's installed?
Amilo21 wrote:
Okay, I'll go for the 2x16GB Ram (am looking at this "https://www.amazon.de/Crucial-Hukommelse-notebook-5200MHz-4800MHz/dp/B0BLTDRRLF/ref=ascdfB0BLTDRRLF?mcid=7e742d19a4a83988899e925a9e2b42fe&language=daDK&tag=dkgoshpadde-21&linkCode=df0&hvadid=718118126117&hvpos=&hvnetw=g&hvrand=423330345103109613&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9192875&hvtargid=pla-1957466101870&gadsource=1&th=1") and a single NVMe SSD. Would you recommend a specific brand, model and how big (GB) it should be or two brands, depending on what I can buy here in EU? Would help a lot, as I don't really know quality brands in this field. Yes, I would very much like a walk through for moving it to the SSD, once I have one installed. Thanks
tomspark wrote:
That Crucial kit's a solid pick (don't worry about the 5200/4800 label, it'll just run at whatever your board supports). For the SSD, any mainstream NVMe from Samsung, WD, or Crucial is the safe bet, just skip the no-name budget brands since the database does steady writes. Good EU options: Samsung 990 EVO or 980, WD Blue SN580 or Black SN770, or Crucial P3 Plus. Size-wise, 500GB is plenty for the apps and databases (your photos and media stay on the big drives); only go 1TB if you want lots of breathing room, it's a small price bump. Grab one of those and ping me once it's installed, I'll walk you through moving SparkBox onto it, step by step.
Amilo21 wrote:
Thanks so far. So the link I already posted and this "https://www.amazon.de/-/en/Crucial-Desktop-Handheld-Consoles-Internal/dp/B0DC8VPSHV/ref=sr11?crid=25XTN5BPL8RN9&dib=eyJ2IjoiMSJ9.ul5sCpbM35WJp-LJxEfNN7qML5Xn1v9lVgw8PrDYSNDtbz6LB-WHBQ-sMe5MdkzezoYmpRtsPhapBYFk1H0b2ikKxKChEEYW7vOpsz2TDGJVEP4g08ecRWuVBv3mJ02UlpT4xbphw6a6cgCVA4147FAGTJByY0rJXyRyB61n5ek1q5Scv40fO7eFBXF9-GVAiTNDIic8WFGvlX-wJaVb841zci1rfs-dzwi6Yiw.OhMDMjy9nC9VUgbnRpKGck-22Evf9xAc2abYdP8PQ&dibtag=se&keywords=Crucial%2BP3%2BPlus&qid=1782651878&sprefix=crucial%2Bp3%2Bplus%2Caps%2C156&sr=8-1&th=1" would be perfect for my uses? Because then I'll order them now and get them sometime next week.
tomspark wrote:
Yep, both are great picks, order away. The Crucial P3 Plus is exactly the kind of drive I'd grab, and that RAM kit pairs perfectly with it. One thing to double-check on the SSD: aim for the 500GB or 1TB size, that's the sweet spot for the apps and databases (no need to go bigger, your photos and media live on the hard drives). Once they arrive and the NVMe's installed, ping me and I'll walk you through moving SparkBox onto it. Enjoy the upgrade.
Amilo21 wrote:
It's been ordered. I'll try and figure out how to ping you when I have them installed π
To be sure, I do just turn off the NAS, unplug its power. Pull out the old 8GB Ram card, insert the two new 16GB Ram and the NVMe and power it on again. Nothing to do before unplugging and nothing actually stored on the 8Gb Ram card?
tomspark wrote:
You've got it right, the RAM holds nothing permanent (everything lives on your drives), so pulling the old stick loses nothing. One tweak to be safe: shut the NAS down properly first, use its power button or the shutdown option and let it fully power off, THEN unplug, rather than pulling the plug while it's still running, that protects the databases. After that: unplug, touch something metal first to zap any static off your hands, swap in the two 16GB sticks, seat the NVMe, plug back in and power on. When it boots back up, just reply right here, that's the perfect way to ping me, and we'll point SparkBox at the new SSD. Enjoy.
Amilo21 wrote:
Thanks. Didn't know the hand metal thing, though I have heard something about the static thing before. I'll write right here then. Thank you very much
tomspark wrote:
Perfect, talk to you then. Enjoy getting the new parts in, it'll make a real difference for Immich.
skylyn wrote:
Just touching metal isn't enough. Touch a metal surface that is grounded. Most likely your NAS has some metal accessible - that will be grounded as long as it's plugged in. Touch that metal before unplugging the NAS, then unplug it and install the RAM and SSD. Note the risk of static electricity is generally over stated, and it's unlikely you'll have enough built up to actually damage a component. Nonetheless, it's better to be safe and take a couple of seconds to touch something grounded to be sure.
tomspark wrote:
Spot on, skylyn, thanks for the correction. Amilo21, that's the precise version: touch the NAS's bare metal case while it's still plugged in (that's what actually grounds you), then pull the plug and do the swap. Like skylyn says the risk is small, but it's a free two-second habit.
kaufman51 wrote:
I was researching this exact topic earlier today. The hardware upgrades are out of my budget at the moment, but would be a fun project in the future.
tomspark wrote:
No rush at all. When you do get to it, the RAM bump alone is the biggest win for Immich, the SSD's a nice bonus on top. Fun little upgrade whenever the budget's there.
Amilo21 wrote:
Thanks mate. I would just have touched some random metal thing. I Touched the NAS turned off, but still plugged in, before i plugged it out and installed the Ram and SSD.
Amilo21 wrote:
I just got the Ram and SSD and have physically put them in the NAS, like skylyn and you suggested. Do I also have to "install" the SSD first, or is putting SParkbox and apps on it part of the "install" process?
tomspark wrote:
Perfect timing β you've done the physical part right, just one quick step before the SparkBox side. Power the NAS on and open the UGOS web interface (same browser page you use to manage it). Look for Storage Manager β the NVMe should show up there automatically. If it prompts you to initialize or format it, go ahead and say yes; that's the NAS making the drive ready to use. It takes about a minute and doesn't touch your existing data or the drives already in the NAS. Once you see the SSD showing up and formatted in UGOS, reply here and I'll walk you through the SparkBox migration step by step.
Amilo21 wrote:
Do I create storage pool or SSD cache? I think cache, but not entirely sure
tomspark wrote:
Go with storage pool, not SSD cache β you've got the right instinct to question it. SSD cache just speeds up your hard drives a little; it doesn't give the NVMe its own space where apps can actually live. Storage pool is what creates that space. Once you create the pool, UGOS will prompt you to create a volume inside it β go ahead and do that, and let it format if it asks. Once you see the volume show up in Storage Manager, reply here and I'll walk you through moving SparkBox onto it.
Amilo21 wrote:
ext4 or btrf? π
I know there was a lot of upsides for my hdd's using btrf and being able to snapshot, but don't know if its the same for ssd?
tomspark wrote:
Go with ext4 β it's the right pick for a drive running SparkBox's apps and databases. Btrfs is excellent on your big hard drives because its snapshot and checksum features protect your photos and media files; on an SSD running app databases it just adds overhead without much real benefit. SparkBox's built-in backups handle the 'snapshot' use case for config anyway. Once the volume's formatted as ext4, reply here and I'll walk you through moving SparkBox onto it.
Amilo21 wrote:
Good I didn't go with my gut, for then I would have created a SSD cache with btrf, if that's even possible. I'm ready to walk through moving Sparkbox to my SSD
tomspark wrote:
Let's do it β you'll need SSH for this part. In UGOS, go to Control Panel β Terminal, tick "Enable SSH service", Apply. Then open Command Prompt (Windows: Win+R, type cmd, Enter) and log in: ssh YOUR-UGOS-USERNAME@YOUR-NAS-IP. Once you're in, first confirm in Storage Manager that your SSD volume says /volume2 (swap that in below if it's different). Then run these in order: sudo sparkbox stop sudo rsync -a /opt/sparkbox/ /volume2/sparkbox/ sudo mv /opt/sparkbox /opt/sparkbox.old && sudo ln -s /volume2/sparkbox /opt/sparkbox sudo sparkbox start Takes a minute or two to copy. When SparkBox comes back up, open the dashboard and let me know what you see!
Amilo21 wrote:
Hey Tom It just won't recognize the first command. Is it supposed to be "sudo sparkbox down" instead?
tomspark wrote:
You got it exactly right β my mistake in that last message. The correct commands are down and up, not stop and start. Here's the corrected sequence: sudo sparkbox down sudo rsync -a /opt/sparkbox/ /volume2/sparkbox/ sudo mv /opt/sparkbox /opt/sparkbox.old && sudo ln -s /volume2/sparkbox /opt/sparkbox sudo sparkbox up Run those in order β takes a minute or two to copy. SparkBox will come back up on the SSD. Let me know what the dashboard shows when it's back!
Amilo21 wrote:
Don't work, maybe because I was to fast and first used the volume2 command, before realizing it was a mistake since my SSD is volume3. Then tried running it again as volume3, but you see here: [OK] SparkBox stopped. Martin@SecretLairNAS:~$ sudo rsync -a /opt/sparkbox/ /volume2/sparkbox/ Martin@SecretLairNAS:~$ sudo rsync -a /opt/sparkbox/ /volume3/sparkbox/ Martin@SecretLairNAS:~$ sudo mv /opt/sparkbox /opt/sparkbox.old && sudo ln -s /v olume3/sparkbox /opt/sparkbox Martin@SecretLairNAS:~$ sudo sparkbox up sudo: sparkbox: command not found Martin@SecretLairNAS:~$ sudo sparkbox up sudo: sparkbox: command not found Martin@SecretLairNAS:~$
Amilo21 wrote:
A small update. Still can't get Sparkbox up again. Tried these things and it looks like the volume3 shortcut was created in addition to the volume2 (which probably should be deleted again). I suspect that it is a UID permission issue again. I have had so many permission issues with my Ugos Pro system, which TomAI have helped solve Updated SSH commands: [OK] SparkBox stopped. Martin@SecretLairNAS:~$ sudo rsync -a /opt/sparkbox/ /volume2/sparkbox/ Martin@SecretLairNAS:~$ sudo rsync -a /opt/sparkbox/ /volume3/sparkbox/ Martin@SecretLairNAS:~$ sudo mv /opt/sparkbox /opt/sparkbox.old && sudo ln -s /v olume3/sparkbox /opt/sparkbox Martin@SecretLairNAS:~$ sudo sparkbox up sudo: sparkbox: command not found Martin@SecretLairNAS:~$ sudo sparkbox up sudo: sparkbox: command not found Martin@SecretLairNAS:~$ sudo sparkbox up sudo: sparkbox: command not found Martin@SecretLairNAS:~$ sudo mv /opt/sparkbox /opt/sparkbox.old && sudo ln -s /volume3/sparkbox /opt/sparkbox Martin@SecretLairNAS:~$ sudo sparkbox up sudo: sparkbox: command not found Martin@SecretLairNAS:~$ ls -l /opt/sparkbox lrwxrwxrwx 1 root root 17 Jul 1 21:11 /opt/sparkbox - /volume3/sparkbox Martin@SecretLairNAS:~$ ls /opt/sparkbox backups CREDITS.md data INSTALL.md knowledge-private MINFROMVERSION qa RELEASESIGNING.md sparkbox support VERSION CONTRIBUTING.md dashboard docs install.sh LICENSE modules README.md scripts state templates Martin@SecretLairNAS:~$ sudo /opt/sparkbox/sparkbox up sudo: /opt/sparkbox/sparkbox: command not found Martin@SecretLairNAS:~$
tomspark wrote:
The symlink and files look right β the most likely culprit is execute permission on the new volume after the copy. Try this: sudo bash /opt/sparkbox/sparkbox up Adding bash in front tells the NAS to run the script directly, which bypasses any permission quirk on the new volume. If SparkBox comes back up, you're done and everything is on the SSD. If that also errors, paste the output of: ls -la /volume3/sparkbox/sparkbox && findmnt /volume3 That shows the exact file permissions and how UGOS mounted the volume so I can narrow it down.
Amilo21 wrote:
Okay, It looked like it started up just fine, but still can't access the dashboard on 8443. All other applications I can access through local network and through remote VPN. Just not the dashboard. Any ideas why that is? / | | | | ) \ \| ' \ / | '| |/ / \ / \ \/ / ) | |) | (| | | | <| |) | () < |/| ./ \,|| ||\\/ \//\\ || SparkBox v1.6.320 - Self-Hosted Privacy Stack Created by Tom Spark | youtube.com/@TomSparkReviews Environment: NAS (generic) [INFO] Starting SparkBox... [INFO] Active modules: adguard core dashboard immich media vpn [INFO] Prepared 19 module config dir(s) for PUID=1000 PGID=10 [+] Running 2/2 β Network sparkboxsbadguard Created 0.4s β Container sb-adguard Started 1.3s [+] Running 3/3 β Container sb-portainer Started 1.6s β Container sb-npm Started 2.2s β Container sb-homepage Started 2.1s [+] Running 1/1 β Container sb-dashboard Started 2.1s [+] Running 5/5 β Network sparkboxsbimmich Created 0.6s β Container sb-immich-db Healthy 3.5s β Container sb-immich-redis Healthy 3.6s β Container sb-immich-ml Started 3.5s β Container sb-immich-server Started 1.5s [+] Running 14/14 β Network sparkboxsbmedia Created 0.8s β Network sparkboxsbmediainternal Created 0.4s β Container sb-bazarr Started 3.4s β Container sb-jellyfin-media Started 3.4s β Container sb-seerr Started 3.4s β Container sb-deunhealth Started 2.0s β Container sb-gluetun Healthy 3.4s β Container sb-sonarr Started 4.5s β Container sb-radarr Started 4.3s β Container sb-sabnzbd Started 4.5s β Container sb-qbittorrent Started 4.5s β Container sb-lidarr Started 4.5s β Container sb-byparr Started 4.5s β Container sb-prowlarr Started 4.5s [+] Running 2/2 β Network sparkboxsbvpn Created 1.2s β Container sb-wg-easy Started 3.3s [OK] SparkBox is running! [INFO] Auto-firing arr-bootstrap in the background (~90-180s)... [INFO] Jellyfin + Seerr credentials will appear in the launcher modals when ready. Service URLs: AdGuard Home: AdGuard Home: http://192.168.1.3:3053 Core Infrastructure: Nginx Proxy Manager: http://192.168.1.3:81 Portainer: http://192.168.1.3:9000 Homepage: http://192.168.1.3:3000 TomSparkBox Dashboard: TomSparkBox Dashboard: http://192.168.1.3:8443 Immich: Immich: http://192.168.1.3:2283 Media Center: qBittorrent: http://192.168.1.3:8089 Prowlarr: http://192.168.1.3:8181 Sonarr: http://192.168.1.3:8989 Radarr: http://192.168.1.3:7878 Bazarr: http://192.168.1.3:6767 Jellyfin (media bundle): http://192.168.1.3:8096 Seerr: http://192.168.1.3:5055 Lidarr: http://192.168.1.3:8686 SABnzbd (Usenet): http://192.168.1.3:8186 FlareSolverr: http://192.168.1.3:8191 Byparr: http://192.168.1.3:8191 Notifiarr: http://192.168.1.3:5454 Remote Access VPN: WireGuard: http://192.168.1.3:51821 Martin@SecretLairNAS:~$
tomspark wrote:
Good sign it started up β let's check if the dashboard container actually came up. Run this: docker ps | grep dashboard If sb-dashboard appears in the list, also run docker logs sb-dashboard --tail 20 and paste what you see β that'll show us if anything went wrong on the new volume. If sb-dashboard isn't listed at all, run: sudo bash /opt/sparkbox/sparkbox up dashboard to bring just that one back up, then try 8443 again.
Amilo21 wrote:
I get this: Martin@SecretLairNAS:~$ sudo docker ps | grep dashboard [sudo] password for Martin: [redacted] c9f9deddd9c2 sparkbox-sparkbox-dashboard "/app/entrypoint.sh β¦" 2 hours ago Up 8 seconds (health: starting) 0.0.0.0:8443-8443/tcp sb-dashboard Martin@SecretLairNAS:~$ docker logs sb-dashboard --tail 20 permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/sb-dashboard/json": dial unix /var/run/docker.sock: connect: permission denied Martin@SecretLairNAS:~$ sudo docker logs sb-dashboard --tail 20 Node.js v20.20.2 node:fs:448 return binding.readFileUtf8(path, stringToFlags(options.flag)); ^ Error: EACCES: permission denied, open '/app/server.js' at Object.readFileSync (node:fs:448:20) at loadSource (node:internal/modules/cjs/loader:1548:17) at Module.extensions..js (node:internal/modules/cjs/loader:1617:44) at Module.load (node:internal/modules/cjs/loader:1266:32) at Module.load (node:internal/modules/cjs/loader:1091:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/runmain:164:12) at node:internal/main/runmainmodule:28:49 { errno: -13, code: 'EACCES', syscall: 'open', path: '/app/server.js' } Node.js v20.20.2 Martin@SecretLairNAS:~$
tomspark wrote:
That error means the dashboard container can't read its own files on the new SSD volume β the rsync copied everything correctly but UGOS's default permissions on the new volume are more restrictive than the old install location. One command fixes it: sudo chmod -R a+rX /volume3/sparkbox/ Then restart with the same command that worked before: sudo bash /opt/sparkbox/sparkbox up After that, 8443 should come up as normal. Let me know what you see!
Amilo21 wrote:
βββΔΌβΚ§ββ: No such device or address Martin@SecretLairNAS:~$ ^C
Amilo21 wrote:
I can find it here Martin@SecretLairNAS:~$ ls -ld /volume3/sparkbox/ drwxr-xr-x 13 root root 4096 Jul 1 19:37 /volume3/sparkbox/ Martin@SecretLairNAS:~$
tomspark wrote:
That 'root root' line tells us everything β UGOS set the new volume to root ownership, so the dashboard can't write to it. One command fixes it: sudo chown -R 1000:1000 /volume3/sparkbox/ Then bring it back up: sudo bash /opt/sparkbox/sparkbox up Give it a minute and try 8443 again. Let me know if the dashboard comes back up!
Amilo21 wrote:
It's up and running now. Thank you very much. I'll try getting TomAI's help with getting Immich running much faster, and perhaps also my mediapipeline that's still not working. If I can't I'll be right back at the forum one of the next days π I'm looking forward to Immich running fast π
tomspark wrote:
Brilliant β enjoy the speed boost! Immich with 32GB behind it should feel completely different to the crash-looping. For the mediapipeline whenever you're ready, just drop back here and I'll take a look with you.