Bastian de Byl
f11391b28f
Caddy was rotating on implicit defaults (100MiB/keep 10/90d) that were not holding -- 20 rotated files per stream and a 190-day-old .gz, 1.3 GB across 16 log streams. Made explicit at 10MiB/keep 3/7d. Note roll_size et al are subdirectives of `output file`, NOT of `log`. Getting that wrong does not degrade gracefully: Caddy refuses to start on a bad config, so every site went down until it was corrected. Worth a `caddy validate` gate before reload. journald had no SystemMaxUse and had reached 4 GB, drifting toward its 10%-of-filesystem default (~190 GB on this root). Capped at 500M. Both are safe to keep short because fluent-bit ships the journal and every Caddy access log into Graylog -- though note its GELF output has been erroring for days, which weakens that premise and wants investigating. The larger find was unrelated to logs: 896 images totalling 59.6 GB with 75% unused (94 tags of greg-time-bot, 73 of fulfillr -- one per deploy) and 5.4 GB of dangling volumes, mostly 804 MB Nextcloud /var/www/html trees orphaned by container recreations. Pruned to 22 images / 15.4 GB, and added a weekly timer keeping 30 days so a rollback still needs no rebuild. Also dropped the decommissioned 6379/tcp redis rule (nothing listening; Immich's redis is on the shared podman network) and the orphaned nosql, s3 and searxng volume dirs. Backup log exclusions turned out to be unnecessary: Gitea logs to console so its log dirs are empty, Nextcloud already excludes its own, BookStack mounts only uploads, and Caddy is not backed up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
34 lines
936 B
YAML
34 lines
936 B
YAML
---
|
|
# Bound the journal. Container stdout all lands here (log_driver=journald) and
|
|
# is shipped to Graylog by fluent-bit, so the local journal only needs to be a
|
|
# buffer -- see the template for the reasoning behind the size.
|
|
- name: ensure journald drop-in directory exists
|
|
become: true
|
|
ansible.builtin.file:
|
|
path: /etc/systemd/journald.conf.d
|
|
state: directory
|
|
owner: root
|
|
group: root
|
|
mode: 0755
|
|
tags: security, service, journald
|
|
|
|
- name: cap journald disk usage
|
|
become: true
|
|
ansible.builtin.template:
|
|
src: journald-size.conf.j2
|
|
dest: /etc/systemd/journald.conf.d/99-size.conf
|
|
owner: root
|
|
group: root
|
|
mode: 0644
|
|
notify: restart_journald
|
|
tags: security, service, journald
|
|
|
|
- name: ensure desired services are started and enabled
|
|
become: true
|
|
ansible.builtin.service:
|
|
name: "{{ item }}"
|
|
state: started
|
|
enabled: true
|
|
loop: "{{ services }}"
|
|
tags: security, service
|