[sudo] password for jasonluaz: [WARN] Docker isn't running — starting it (systemctl start docker)... [ERROR] Docker is not reachable, so no apps can be started. [ERROR] Check it with: systemctl status docker (and: journalctl -u docker -n 20) [ERROR] The usual cause after an outage is a 100%-full disk — check: df -h [ERROR] Once Docker is back: sudo sparkbox up jasonluaz@server1:~$
19 replies
Chris wrote:
Got it — this is an automatic acknowledgment from Chris. Your post is in the support queue, and we’ll get to it as soon as we can. Legend members receive priority replies. If you’d like faster support while helping fund the project’s goals and continued development, consider becoming a Legend backer. <!-- chris-support-queue-ack:v1 --
Chris wrote:
Thanks — that confirms Docker itself is reachable again. In the same SSH window, run sudo sparkbox up once; SparkBox’s recovery message directs this after Docker is back. When it finishes, refresh the dashboard. If a particular app still will not open, reply with that app’s name and the exact error only.
Chris wrote:
Good news on isolating this: that raw-IP test still timing out (curl exit 28, HTTP 000) even though the public-resolver DNS lookup worked confirms this is NOT a DNS problem -- something is blocking all outbound container traffic on this box, on both sbproxy and the default bridge, even after repair-network already ran and even with DEFAULTFORWARDPOLICY=ACCEPT confirmed. That rules out every fix we currently have automated for Windows/WSL (the container-DNS repair doesn't apply here, and UFW forwarding is fine), and it matches another customer's Windows/WSL2 case we're already tracking with the identical signature. This looks like a genuine gap on our side rather than something in your setup, so I've filed it for engineering with both cases' evidence. Please don't run repair-network again or change Windows/Hyper-V firewall settings for now -- neither is verified to help, and we don't want to send you in circles. I'll follow up here as soon as there's a real diagnosis or a fix to test. Sorry for the wait, and thanks for running that test cleanly, it genuinely narrowed things down.
Chris wrote:
Honest answer: I can't give you a firm date yet, but I can tell you exactly where things stand. Both problems you hit are confirmed real bugs on our side — one is the repair-network command hanging for hours (two of its checks have no timeout on a box like yours), and the other is that nothing in SparkBox today can even detect the Windows-side block that is stopping your containers' internet, so it kept recommending steps that can't fix it. Fixes for both are written and are now going through the review pipeline, but neither is in a released version yet — the current release (1.6.670) does not include them. The moment a release that includes them is published, I'll reply right here with the version number and the exact next step. I genuinely can't promise a date before then. Nothing is needed from you in the meantime, and please don't re-run repair-network — it can't help with this one. Thanks again for running that test so cleanly; it's exactly what let us pin both of these down.
Chris wrote:
This is a Docker start problem rather than your password. In the server’s SSH window, run sudo sparkbox doctor and reply with only the [Docker] and [System Resources] results—please leave out passwords and keys. That will show whether the next safe step is a Docker-service issue or a full disk.
jasonluaz wrote:
[Docker] [OK] Docker daemon is accessible [OK] Docker version: 29.8.0 (engine) [OK] Docker Compose: 5.5.1 [Docker Socket] [OK] Docker socket exists at /var/run/docker.sock [OK] Docker socket is readable
jasonluaz wrote:
[Container Egress] [WARN] The apps' network (sbproxy) cannot reach tomsparkbox.com. This is the network every SparkBox app runs on, including the reverse proxy — so anything that fetches from the internet (certificates, metadata, indexers, updates) will fail. It also stops the reverse proxy (sb-npm) finishing its startup: NPM downloads an IP-range list from Amazon the moment it boots, and with no way out that download never answers, so it waits there forever. The container stays 'Up (unhealthy)' and its log ends mid-line at 'Fetching IP Ranges from online services'. Nothing is wrong with NPM; fix egress below and it finishes booting. To fix container egress: 0. Run: sudo sparkbox repair-network — on Windows/WSL this usually IS the fix: WSL gives containers a DNS server only the host can reach, so apps fail with 'bad address' / 'can't reach this page' while your terminal is online. 1. UFW: grep DEFAULTFORWARDPOLICY /etc/default/ufw (should be ACCEPT) 2. Run: sudo sparkbox repair-network (re-tests egress itself; exits without touching your apps if the network is not the problem) 3. If you have a custom /etc/docker/daemon.json, check its DNS settings. Re-run 'sparkbox doctor' to verify. [WARN] Docker's default bridge cannot reach tomsparkbox.com. What this breaks: the dashboard's one-click Update button. It downloads from inside a helper container on that bridge, so it fails with 'Could not resolve host: get.tomsparkbox.com'. Optional Pro-tier license activation won't work either. Updating over SSH still works — that downloads on the host: sudo sparkbox upgrade Same fixes as above (UFW forward policy, sparkbox repair-network).
Anthony wrote:
Thanks — this confirms a real outbound-connectivity failure from both Docker networks, which can leave NPM waiting during startup. It does not by itself identify a UFW or Windows firewall cause. Before running a repair that may briefly restart the apps, please run this read-only batch in the same SSH session and paste only its output: sudo sparkbox --version uname -r docker info --format '{{.OperatingSystem}}' The first line gives the installed SparkBox version; a kernel containing microsoft indicates WSL, and Docker Desktop in the last line identifies Docker Desktop. Stop there for now — please do not change UFW/Docker DNS or run repair-network yet.
jasonluaz wrote:
SparkBox v1.6.658 6.18.33.2-microsoft-standard-WSL2 Ubuntu 24.04.4 LTS Not sure if this is how I was supposed to do that
Anthony wrote:
That's exactly right, no worries. That confirms you're on Windows using WSL with Docker's own engine (not Docker Desktop) — this is a known pattern on that setup. Next step, same SSH window: sudo sparkbox repair-network This takes your apps offline for about a minute while it rebuilds the network. When it finishes it will say either "nothing to repair" (means something else is blocking it) or that the internet connection is back. If it says fixed, refresh the dashboard and the Update button should work. If it still fails, paste the last few lines it printed.
jasonluaz wrote:
Its still running, should it be?
Anthony wrote:
No -- repair-network normally finishes in about a minute, so something is stuck. Please do NOT press Ctrl-C on that window; instead open a SECOND SSH window and run these two read-only checks: (1) docker ps -a --filter name=sb- to see which containers are up/created/restarting, and (2) paste the last ~10 lines currently shown in the ORIGINAL window that is still running. That will show whether it is stuck pulling an image or waiting on the network, and we will pick the next step from there.
jasonluaz wrote:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9e6bd2e325b4 jc21/nginx-proxy-manager:2.15.1 "/init" 4 hours ago Up 4 hours (unhealthy) 0.0.0.0:80-81-80-81/tcp, [::]:80-81-80-81/tcp, 0.0.0.0:443-443/tcp, [::]:443-443/tcp sb-npm 5e8adafe6b60 portainer/portainer-ce:2.45.0 "/portainer --admin-…" 4 hours ago Up 4 hours 8000/tcp, 9443/tcp, 0.0.0.0:9000-9000/tcp sb-portainer bd2404da8de4 ghcr.io/gethomepage/homepage:v1.13.2 "docker-entrypoint.s…" 4 hours ago Up 4 hours (healthy) 0.0.0.0:3000-3000/tcp, [::]:3000-3000/tcp sb-homepage root@server1:~
Chris wrote:
Thanks for sticking with this. A couple of things from that output: 1. It's safe to stop the still-running window now. Your apps (npm, portainer, homepage) already came back up per that docker ps, so the risky part of repair-network already finished -- go ahead and press Ctrl-C there. 2. sb-npm is still showing "Up 4 hours (unhealthy)", which means the underlying internet-access problem is not fixed yet even after the rebuild. 3. We also traced why that window never returned: two of repair-network's internal checks have no timeout on a box like yours where the network is genuinely broken, so they can hang indefinitely instead of failing fast. That's a real bug on our side and we've logged it for a fix -- it is not something you did wrong. Next, in a fresh SSH window (read-only, nothing restarts): sudo sparkbox doctor grep DEFAULTFORWARDPOLICY /etc/default/ufw Please paste the [Container Egress] section of the doctor output plus that grep line. If DEFAULTFORWARDPOLICY is not ACCEPT, that is likely why containers still cannot reach the internet, and we will walk you through the fix from there.
jasonluaz wrote:
[Container Egress] [WARN] The apps' network (sbproxy) cannot reach tomsparkbox.com. This is the network every SparkBox app runs on, including the reverse proxy — so anything that fetches from the internet (certificates, metadata, indexers, updates) will fail. It also stops the reverse proxy (sb-npm) finishing its startup: NPM downloads an IP-range list from Amazon the moment it boots, and with no way out that download never answers, so it waits there forever. The container stays 'Up (unhealthy)' and its log ends mid-line at 'Fetching IP Ranges from online services'. Nothing is wrong with NPM; fix egress below and it finishes booting. To fix container egress: 0. Run: sudo sparkbox repair-network — on Windows/WSL this usually IS the fix: WSL gives containers a DNS server only the host can reach, so apps fail with 'bad address' / 'can't reach this page' while your terminal is online. 1. UFW: grep DEFAULTFORWARDPOLICY /etc/default/ufw (should be ACCEPT) 2. Run: sudo sparkbox repair-network (re-tests egress itself; exits without touching your apps if the network is not the problem) 3. If you have a custom /etc/docker/daemon.json, check its DNS settings. Re-run 'sparkbox doctor' to verify. [WARN] Docker's default bridge cannot reach tomsparkbox.com. What this breaks: the dashboard's one-click Update button. It downloads from inside a helper container on that bridge, so it fails with 'Could not resolve host: get.tomsparkbox.com'. Optional Pro-tier license activation won't work either. Updating over SSH still works — that downloads on the host: sudo sparkbox upgrade Same fixes as above (UFW forward policy, sparkbox repair-network). [Port Conflicts] [OK] Port 80: in use (Docker -- likely SparkBox) [OK] Port 443: in use by SparkBox (sb-npm) [OK] Port 8443 (the SparkBox dashboard itself): available [OK] Port 9000: in use by SparkBox (sb-portainer) [Portainer Auth] [ERROR] sb-portainer IS listening on 127.0.0.1:9000, but the connection timed out instead of connecting — that's Windows/WSL2's 'mirrored' networking mode blackholing loopback traffic, not a Portainer problem The tell: a genuinely closed port refuses instantly; this one hangs for several seconds because a Hyper-V firewall rule (or the mirrored-loopback bug itself) is silently dropping the connection instead of rejecting it. This blocks every app on this box the same way, not just Portainer — the dashboard and qBittorrent too. Fix from an administrator PowerShell: Set-NetFirewallHyperVVMSetting -Name '{40E0AC32-46A5-438A-A0B2-2B479E8F2E90}' -DefaultInboundAction Allow wsl --shutdown Still blocked? Do NOT switch back to NAT mode — that only moves the fault: NAT hands the box a private 172.x address that no phone, TV or console can reach, and this same doctor will then flag it and point you back to mirrored, which is the loop you are stuck in. Fix the firewall rule above, or use the NAT + port-forward walkthrough below instead of flipping modes. Full walkthrough: https://tomsparkbox.com/guides/run-sparkbox-on-windows-wsl [WARN] Windows Firewall may have your network on its 'Public' profile, which blocks every incoming connection to this box Windows flips a network to Public silently (a new router, a Windows update, a changed port), and while it is Public the firewall drops inbound traffic before it reaches WSL — so app addresses time out from the PC itself no matter whether WSL is mirrored or NAT, and even after the Hyper-V rule is set. Check from an administrator PowerShell: Get-NetConnectionProfile | Select-Object Name, NetworkCategory If your home network says 'Public', flip it back to Private: Set-NetConnectionProfile -NetworkCategory Private Then reload the app address on the PC. [VPN Tunnel] [INFO] sb-gluetun not deployed — media stack not installed (or VPN module skipped) [Hardware Transcoding] [INFO] /dev/dri not found -- Intel/AMD hardware transcoding not available [Media Storage] [OK] MEDIAROOT=/opt/sparkbox/data/media exists [OK] MEDIAROOT is writable [OK] Your apps (user id 1000) can read your media library [INFO] Free space at MEDIAROOT: 950GB [System Resources] [INFO] Total RAM: 3709MB [INFO] Available RAM: 2880MB [OK] 3709MB RAM is enough for 2 enabled apps (~1024MB estimated) [OK] No out-of-memory kills in the last 24h [INFO] Free disk at /opt/sparkbox (the SparkBox install disk): 950GB [Container Health] [OK] All SparkBox containers are running and passing their health checks [OK] Zombie processes: 0 [OK] Load average is fine for 8 core(s) [WARN] sb-npm never finished starting: it is parked on the IP-range list it downloads at boot, because this box's containers can't reach the internet. It is not broken and its data is fine — it is waiting. This is the same fault as the Container Egress warning higher up. Fix the internet access first: 1. grep DEFAULTFORWARDPOLICY /etc/default/ufw (should be ACCEPT) 2. sudo sparkbox repair-network Then restart the proxy: sudo sparkbox restart core [Windows (WSL) Networking] [OK] Running on Windows (WSL) with 10.14.0.2 — a real home-network address, so other devices can reach this box [WARN] Windows Firewall may have your network on its 'Public' profile, which blocks every incoming connection to this box Windows flips a network to Public silently (a new router, a Windows update, a changed port), and while it is Public the firewall drops inbound traffic before it reaches WSL — so app addresses time out from the PC itself no matter whether WSL is mirrored or NAT, and even after the Hyper-V rule is set. Check from an administrator PowerShell: Get-NetConnectionProfile | Select-Object Name, NetworkCategory If your home network says 'Public', flip it back to Private: Set-NetConnectionProfile -NetworkCategory Private Then reload the app address on the PC. [WARN] No start-with-Windows task found — this box shuts down about a minute after you close its last window, and nothing brings it back after a reboot WSL stops the Linux machine ~60s after the last Ubuntu/PowerShell window closes (systemd does not keep it alive), and it stays down after a restart. A scheduled task that holds it open is the fix. From an ADMINISTRATOR PowerShell on Windows, paste this as one line: Register-ScheduledTask -Force -TaskName "SparkBox" -Action (New-ScheduledTaskAction -Execute "C:\Windows\System32\wsl.exe" -Argument "-d Ubuntu --exec sleep infinity") -Trigger (New-ScheduledTaskTrigger -AtStartup) -Principal (New-ScheduledTaskPrincipal -UserId $env:USERNAME -LogonType S4U) -Settings (New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -ExecutionTimeLimit ([TimeSpan]::Zero)) then start it once without rebooting: Start-ScheduledTask -TaskName "SparkBox" (If your distro isn't 'Ubuntu', change the -d part; 'wsl -l' lists the names.) Guide: https://tomsparkbox.com/guides/run-sparkbox-on-windows-wsl.htmlautostart [Network] [OK] Saved address matches this box [Internet exposure] [OK] Admin surfaces are open to your local network only (this host's internet route uses a private address: 10.14.0.2): portainer (port 9000) ============================================ 5 issue(s) found. Review warnings above and fix as needed. ============================================ DEFAULTFORWARDPOLICY="ACCEPT"
Chris wrote:
That DEFAULTFORWARDPOLICY=ACCEPT line rules out the UFW cause, and the doctor output confirms container egress (sbproxy and the default bridge) is still blocked even after the rebuild. One thing to clear up first: repair-network running for hours without returning is a real bug on our side (two of its internal checks had no timeout when egress is genuinely blocked) - we've logged that separately, it is not something you did or need to retry. To narrow down what is actually blocking egress, please run these two read-only checks (nothing here changes settings or restarts anything) and paste both results: docker run --rm --network sbproxy curlimages/curl:latest -sS -o /dev/null -w '%{httpcode}\n' -m 10 https://1.1.1.1 docker run --rm --dns 1.1.1.1 alpine:3 nslookup get.tomsparkbox.com The first talks to an IP directly, bypassing DNS entirely - if it also times out, this is not a DNS problem and something is blocking all outbound container traffic, which points away from UFW toward Windows-side networking. The second tests name resolution using a public resolver instead of the one WSL hands your containers - if it succeeds, that confirms DNS is the specific broken link and we can fix it directly.
jasonluaz wrote:
root@server1:~ docker run --rm --network sbproxy curlimages/curl:latest -sS -o /dev/null -w '%{httpcode}\n' -m 10 https://1.1.1.1 docker run --rm --dns 1.1.1.1 alpine:3 nslookup get.tomsparkbox.com 000 curl: (28) Connection timed out after 10000 milliseconds Server: 1.1.1.1 Address: 1.1.1.1:53 Non-authoritative answer: Name: get.tomsparkbox.com Address: 172.67.213.11 Name: get.tomsparkbox.com Address: 104.21.61.187 Non-authoritative answer: Name: get.tomsparkbox.com Address: 2606:4700:3032::6815:3dbb Name: get.tomsparkbox.com Address: 2606:4700:3035::ac43:d50b root@server1:~
jasonluaz wrote:
Oh and by the way thanks for all this help. All this is a foreign language to me lol.
jasonluaz wrote:
Any idea how long a fix will take, any rough estimate?