Files
deploy_home/ansible/roles/podman/tasks/containers/home/photos.yml
T
Bastian de BylandClaude Opus 5 ae99ba415e fix(immich): keep the ML models resident, and pin the ML URL explicitly
Smart search, face detection and OCR were all failing with

  Machine learning request to "http://immich-machine-learning:3003" failed

while the container still reported Up. Podman only sees PID 1: gunicorn's
master was alive and holding the listening socket, but its worker had died at a
WORKER TIMEOUT and was never respawned -- hence a connect timeout rather than a
refusal. The dead worker was a zombie whose remaining thread was stuck in
uninterruptible sleep in exit_mmap, so it survived SIGKILL, podman rm -f and
rm -f -t 0, and kept the container name and network alias until the host was
rebooted.

MACHINE_LEARNING_MODEL_TTL=0 addresses the cause rather than the symptom. The
default unloads models after 300s idle, so every search following a gap
reloaded four of them (CLIP, buffalo_l detection + recognition, PP-OCRv5) on a
CPU-only 4-core box and then tore those mappings back down -- and that teardown
is what wedged. Keeping them resident costs ~1-2 GB and removes the path.

IMMICH_MACHINE_LEARNING_URL is now set explicitly instead of relying on
immich's implicit default, so the ML container can be renamed without silently
losing search, and the name is a variable so a replacement can be stood up
beside a broken one without editing tasks.

Verified after deploy: zero ML failures, "in-memory cache with unloading
disabled", ping 200 from immich-server, 26/26 containers healthy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-28 11:39:57 -04:00

212 lines
6.9 KiB
YAML

---
- name: create required photos volumes
become: true
ansible.builtin.file:
path: "{{ item }}"
state: directory
owner: "{{ podman_subuid.stdout }}"
group: "{{ podman_user }}"
mode: 0755
notify: restorecon podman
loop:
- "{{ photos_path }}/psql"
- "{{ photos_path }}/mlcache"
- "{{ photos_path }}/storage"
- "{{ photos_path }}/immich"
- name: flush handlers
ansible.builtin.meta: flush_handlers
- name: create cifs credentials directory
become: true
ansible.builtin.file:
path: /etc/cifs
state: directory
owner: root
group: root
mode: 0700
# /etc/fstab is 0644 by design, so an inline `password=` is readable by every
# local user. Keep the credential in a 0600 file and reference it instead.
- name: deploy photos cifs credentials
become: true
ansible.builtin.template:
src: cifs-credentials.j2
dest: /etc/cifs/photos.creds
owner: root
group: root
mode: 0600
vars:
cifs_username: photos
cifs_password: "{{ photos_cifs_pass }}"
no_log: true
# These mounts previously failed permanently whenever TrueNAS was power-cycled:
# a systemd .mount unit does NOT retry after a failed attempt, so the share came
# back and nothing remounted, leaving immich serving an empty library while its
# database still referenced 14k assets.
#
# x-systemd.automount the actual fix -- any ACCESS to the path re-attempts the
# mount, so a failure stops being terminal
# _netdev / nofail order after network-online; a dead NAS must not block boot
# soft fail I/O with an error instead of blocking forever, so
# immich-server can be restarted during an outage rather
# than wedging in uninterruptible sleep. Accepted trade-off:
# a write interrupted mid-flight fails and is retried.
# idle-timeout unmount when unused, which clears stale handles instead
# of nursing a half-dead connection
# resilienthandles SMB3 rides out brief server blips transparently
#
# See also cifs-watchdog.sh.j2, which restarts immich-server when a mount that
# was unhealthy becomes healthy again -- the automount restores the FILESYSTEM,
# but the container still holds the old, empty view until it is bounced.
- name: mount photos cifs
become: true
ansible.posix.mount:
src: "{{ photos_cifs_src }}"
path: "{{ photos_path }}/storage"
fstype: cifs
opts: "{{ cifs_mount_opts }}"
state: mounted
- name: mount immich cifs
become: true
ansible.posix.mount:
src: "{{ immich_cifs_src }}"
path: "{{ photos_path }}/immich"
fstype: cifs
opts: "{{ cifs_mount_opts }}"
state: mounted
# systemd-fstab-generator creates the .automount unit from x-systemd.automount,
# but ansible.posix.mount mounts the path directly and never starts it, leaving
# the on-access trigger INACTIVE -- so a dropped mount stayed dropped, which is
# the exact failure this work exists to fix. Enable it explicitly.
- name: enable cifs automount units
become: true
ansible.builtin.systemd:
name: "{{ item }}"
enabled: true
state: started
daemon_reload: true
loop: "{{ cifs_watchdog_mounts | map('regex_replace', '^/', '') | map('regex_replace', '/', '-') | map('regex_replace', '$', '.automount') | list }}"
failed_when: false
- import_tasks: podman/podman-check.yml
vars:
container_name: "{{ immich_ml_container }}"
container_image: "{{ ml_image }}"
# MACHINE_LEARNING_MODEL_TTL=0 keeps the models resident instead of unloading
# them after 300s idle. The default made every search following an idle gap
# reload four models (CLIP, buffalo_l detection + recognition, PP-OCRv5) on a
# CPU-only 4-core box, then tear those mappings down again -- and it is that
# teardown, in exit_mmap, that wedged a worker thread in uninterruptible sleep
# on 2026-08-27 and took smart search, face detection and OCR down with it.
# Costs ~1-2 GB resident to remove the code path entirely.
- name: create immich-ml container
become: true
become_user: "{{ podman_user }}"
containers.podman.podman_container:
name: "{{ immich_ml_container }}"
image: "{{ ml_image }}"
restart_policy: on-failure:3
log_driver: journald
network:
- shared
env:
MACHINE_LEARNING_MODEL_TTL: "0"
volumes:
- "{{ photos_path }}/mlcache:/cache"
- name: create systemd startup job for immich-machine-learning
include_tasks: podman/systemd-generate.yml
vars:
container_name: "{{ immich_ml_container }}"
- import_tasks: podman/podman-check.yml
vars:
container_name: immich-redis
container_image: "{{ redis_image }}"
- name: create immich-redis container
become: true
become_user: "{{ podman_user }}"
containers.podman.podman_container:
name: immich-redis
image: "{{ redis_image }}"
restart_policy: on-failure:3
log_driver: journald
network:
- shared
- name: create systemd startup job for immich-redis
include_tasks: podman/systemd-generate.yml
vars:
container_name: immich-redis
- import_tasks: podman/podman-check.yml
vars:
container_name: immich-postgres
container_image: "{{ db_image }}"
- name: create immich-db container
become: true
become_user: "{{ podman_user }}"
containers.podman.podman_container:
name: immich-postgres
image: "{{ db_image }}"
restart_policy: on-failure:3
log_driver: journald
network:
- shared
env:
POSTGRES_DATABASE: photos
POSTGRES_USER: photos
POSTGRES_PASSWORD: "{{ photos_db_pass }}"
POSTGRES_INITDB_ARGS: "--data-checksums"
volumes:
- "{{ photos_path }}/psql:/var/lib/postgresql/data"
- name: create systemd startup job for immich-postgres
include_tasks: podman/systemd-generate.yml
vars:
container_name: immich-postgres
- import_tasks: podman/podman-check.yml
vars:
container_name: immich-server
container_image: "{{ image }}"
- name: create immich container
become: true
become_user: "{{ podman_user }}"
containers.podman.podman_container:
name: immich-server
image: "{{ image }}"
restart_policy: on-failure:3
log_driver: journald
network:
- shared
env:
REDIS_HOSTNAME: immich-redis
DB_HOSTNAME: immich-postgres
DB_DATABASE_NAME: photos
DB_USERNAME: photos
DB_PASSWORD: "{{ photos_db_pass }}"
IMMICH_PORT: 8088
# Set explicitly rather than relying on immich's default of
# http://immich-machine-learning:3003, so the ML container can be renamed
# without silently losing search, faces and OCR.
IMMICH_MACHINE_LEARNING_URL: "http://{{ immich_ml_container }}:3003"
volumes:
- "{{ photos_path }}/storage:/mnt/media/originals"
- "{{ photos_path }}/immich:/usr/src/app/upload"
ports:
- "8088:8088"
- name: create systemd startup job for immich-server
include_tasks: podman/systemd-generate.yml
vars:
container_name: immich-server