fix(immich): keep the ML models resident, and pin the ML URL explicitly

Smart search, face detection and OCR were all failing with

  Machine learning request to "http://immich-machine-learning:3003" failed

while the container still reported Up. Podman only sees PID 1: gunicorn's
master was alive and holding the listening socket, but its worker had died at a
WORKER TIMEOUT and was never respawned -- hence a connect timeout rather than a
refusal. The dead worker was a zombie whose remaining thread was stuck in
uninterruptible sleep in exit_mmap, so it survived SIGKILL, podman rm -f and
rm -f -t 0, and kept the container name and network alias until the host was
rebooted.

MACHINE_LEARNING_MODEL_TTL=0 addresses the cause rather than the symptom. The
default unloads models after 300s idle, so every search following a gap
reloaded four of them (CLIP, buffalo_l detection + recognition, PP-OCRv5) on a
CPU-only 4-core box and then tore those mappings back down -- and that teardown
is what wedged. Keeping them resident costs ~1-2 GB and removes the path.

IMMICH_MACHINE_LEARNING_URL is now set explicitly instead of relying on
immich's implicit default, so the ML container can be renamed without silently
losing search, and the name is a variable so a replacement can be stood up
beside a broken one without editing tasks.

Verified after deploy: zero ML failures, "in-memory cache with unloading
disabled", ping 200 from immich-server, 26/26 containers healthy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Bastian de Byl
2026-08-28 11:39:57 -04:00
co-authored by Claude Opus 5
parent 2a6390c0ad
commit ae99ba415e
2 changed files with 24 additions and 3 deletions
+8
View File
@@ -27,6 +27,14 @@ hass_path: "{{ podman_volumes }}/hass"
partsy_path: "{{ podman_volumes }}/partsy"
partsy_skudak_path: "{{ podman_volumes }}/partsy-skudak"
photos_path: "{{ podman_volumes }}/photos"
# Named rather than hardcoded so the ML service can be stood up beside a broken
# one without editing tasks. On 2026-08-28 a worker thread wedged in
# uninterruptible sleep (D state) in exit_mmap, which made the container
# unremovable -- it survived SIGKILL and `podman rm -f` and kept its name and
# network alias until the host was rebooted. The reboot cleared it, so this
# stays on the canonical name; see MACHINE_LEARNING_MODEL_TTL in photos.yml for
# the change that stops it recurring.
immich_ml_container: immich-machine-learning
uptime_kuma_path: "{{ podman_volumes }}/uptime-kuma"
uptime_kuma_personal_path: "{{ podman_volumes }}/uptime-kuma-personal"
zomboid_path: "{{ podman_volumes }}/zomboid"