Bastian de BylandClaude Opus 5
eea55def6c
Graylog was the worst cost/benefit tenant on this 4-core box: two JVMs plus
MongoDB holding ~1.6 GB resident and ~3% CPU around the clock to store ~3k
messages a day -- about 28 MB across its four live indices. journald already
retains ~25 days of the same logs at its 500M cap, so this costs searchability,
not the logs.
The switch is `graylog_enabled` in inventory rather than a role default,
because three roles read it (common, podman, graylog-config). The disabled
path is an active teardown, not a skipped create: the containers already on
the host keep running and their systemd user units keep restarting them at
boot unless something stops and removes them. fluent-bit follows the same
flag -- with the GELF sink down it would spin retrying a dead 127.0.0.1:12202
and fill the journal it exists to drain -- but only the service state follows,
so re-enabling is a restart rather than a reinstall.
Caddy reloads were silently no-ops. The handler read /etc/caddy/Caddyfile,
which is a single-file bind mount, and podman binds those by inode; the
template module writes a temp file and renames it into place, so every deploy
gave the host file a new inode while the container kept seeing the one it was
created with. Config changes only ever landed when something recreated the
container. {{ caddy_path }}/config is also mounted, as a *directory*, and
directory mounts resolve names at open() time -- so /config/Caddyfile is
always the file Ansible just wrote.
awsddns and its four siblings had accumulated 12 zombies over 30 days of
uptime. The image's PID 1 is busybox crond, which only waitpid()s the job PIDs
it tracks and does no generic orphan reaping, so whenever the run-parts/sh
layer exited before the script it left a permanent <defunct>. init: true puts
catatonit at PID 1 to reap them, and the recreation clears the existing ones.
Also bumps fulfillr and greg-time-bot images.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
154 lines
5.0 KiB
YAML
154 lines
5.0 KiB
YAML
---
|
|
# The image's PID 1 is busybox crond (ENTRYPOINT crond -f -d 8), which runs the
|
|
# /etc/periodic/15min/awsddns script every quarter hour. busybox crond only
|
|
# waitpid()s the job PIDs it is tracking -- it does no generic orphan reaping --
|
|
# so whenever the run-parts/sh layer between crond and the script exits first,
|
|
# the script is reparented to PID 1 and stays <defunct> forever. That had left
|
|
# 12 zombies across these five containers after 30 days of uptime.
|
|
#
|
|
# init: true makes podman inject catatonit as PID 1 with crond as its child, and
|
|
# catatonit reaps every orphan it inherits. Changing this recreates the
|
|
# containers, which also clears the zombies already accumulated.
|
|
- import_tasks: podman/podman-check.yml
|
|
vars:
|
|
container_name: awsddns
|
|
container_image: "{{ image }}"
|
|
|
|
- name: create home.debyl.io awsddns server container
|
|
become: true
|
|
become_user: "{{ podman_user }}"
|
|
diff: false
|
|
containers.podman.podman_container:
|
|
name: awsddns
|
|
image: "{{ image }}"
|
|
restart_policy: on-failure:3
|
|
log_driver: journald
|
|
init: true
|
|
env:
|
|
AWS_ZONE_TTL: 60
|
|
AWS_ZONE_ID: "{{ aws_zone_id }}"
|
|
AWS_ZONE_HOSTNAME: "{{ aws_zone_hostname }}"
|
|
AWS_ACCESS_KEY_ID: "{{ aws_access_key_id }}"
|
|
AWS_SECRET_ACCESS_KEY: "{{ aws_secret_access_key }}"
|
|
AWS_DEFAULT_REGION: "{{ aws_default_region }}"
|
|
|
|
- name: create systemd startup job for awsddns
|
|
include_tasks: podman/systemd-generate.yml
|
|
vars:
|
|
container_name: awsddns
|
|
|
|
- import_tasks: podman/podman-check.yml
|
|
vars:
|
|
container_name: awsddns-skudak
|
|
container_image: "{{ image }}"
|
|
|
|
- name: create wiki.skudakrennsport.com awsddns server container
|
|
become: true
|
|
become_user: "{{ podman_user }}"
|
|
diff: false
|
|
containers.podman.podman_container:
|
|
name: awsddns-skudak
|
|
image: "{{ image }}"
|
|
restart_policy: on-failure:3
|
|
log_driver: journald
|
|
init: true
|
|
env:
|
|
AWS_ZONE_TTL: 60
|
|
AWS_ZONE_ID: "{{ aws_skudak_zone_id }}"
|
|
AWS_ZONE_HOSTNAME: "{{ aws_skudak_zone_hostname }}"
|
|
AWS_ACCESS_KEY_ID: "{{ aws_skudak_access_key_id }}"
|
|
AWS_SECRET_ACCESS_KEY: "{{ aws_skudak_secret_access_key }}"
|
|
AWS_DEFAULT_REGION: "{{ aws_default_region }}"
|
|
|
|
- name: create systemd startup job for awsddns-skudak
|
|
include_tasks: podman/systemd-generate.yml
|
|
vars:
|
|
container_name: awsddns-skudak
|
|
|
|
- import_tasks: podman/podman-check.yml
|
|
vars:
|
|
container_name: awsddns-fulfillr
|
|
container_image: "{{ image }}"
|
|
|
|
- name: create fulfillr.debyltech.com awsddns server container
|
|
become: true
|
|
become_user: "{{ podman_user }}"
|
|
diff: false
|
|
containers.podman.podman_container:
|
|
name: awsddns-fulfillr
|
|
image: "{{ image }}"
|
|
restart_policy: on-failure:3
|
|
log_driver: journald
|
|
init: true
|
|
env:
|
|
AWS_ZONE_TTL: 60
|
|
AWS_ZONE_ID: "{{ fulfillr_zone_id }}"
|
|
AWS_ZONE_HOSTNAME: "{{ fulfillr_server_name }}"
|
|
AWS_ACCESS_KEY_ID: "{{ fulfillr_dns_access_key }}"
|
|
AWS_SECRET_ACCESS_KEY: "{{ fulfillr_dns_secret_key }}"
|
|
AWS_DEFAULT_REGION: "{{ fulfillr_region }}"
|
|
|
|
- name: create systemd startup job for awsddns-fulfillr
|
|
include_tasks: podman/systemd-generate.yml
|
|
vars:
|
|
container_name: awsddns-fulfillr
|
|
|
|
- import_tasks: podman/podman-check.yml
|
|
vars:
|
|
container_name: awsddns-fulfillr-dev
|
|
container_image: "{{ image }}"
|
|
|
|
# Staging back-office DNS — same Route53 zone + creds as prod fulfillr, just a
|
|
# different hostname (-> same host IP; Caddy routes both by Host header).
|
|
- name: create fulfillr-dev.debyltech.com awsddns server container
|
|
become: true
|
|
become_user: "{{ podman_user }}"
|
|
diff: false
|
|
containers.podman.podman_container:
|
|
name: awsddns-fulfillr-dev
|
|
image: "{{ image }}"
|
|
restart_policy: on-failure:3
|
|
log_driver: journald
|
|
init: true
|
|
env:
|
|
AWS_ZONE_TTL: 60
|
|
AWS_ZONE_ID: "{{ fulfillr_zone_id }}"
|
|
AWS_ZONE_HOSTNAME: "{{ fulfillr_dev_server_name }}"
|
|
AWS_ACCESS_KEY_ID: "{{ fulfillr_dns_access_key }}"
|
|
AWS_SECRET_ACCESS_KEY: "{{ fulfillr_dns_secret_key }}"
|
|
AWS_DEFAULT_REGION: "{{ fulfillr_region }}"
|
|
|
|
- name: create systemd startup job for awsddns-fulfillr-dev
|
|
include_tasks: podman/systemd-generate.yml
|
|
vars:
|
|
container_name: awsddns-fulfillr-dev
|
|
|
|
- import_tasks: podman/podman-check.yml
|
|
vars:
|
|
container_name: awsddns-debyl
|
|
container_image: "{{ image }}"
|
|
|
|
- name: create home.debyl.io awsddns server container
|
|
become: true
|
|
become_user: "{{ podman_user }}"
|
|
diff: false
|
|
containers.podman.podman_container:
|
|
name: awsddns-debyl
|
|
image: "{{ image }}"
|
|
restart_policy: on-failure:3
|
|
log_driver: journald
|
|
init: true
|
|
env:
|
|
AWS_ZONE_TTL: 60
|
|
AWS_ZONE_ID: "Z07501202A6AYMHCVP50A"
|
|
AWS_ZONE_HOSTNAME: "home.debyl.io"
|
|
AWS_ACCESS_KEY_ID: "{{ aws_access_key_id }}"
|
|
AWS_SECRET_ACCESS_KEY: "{{ aws_secret_access_key }}"
|
|
AWS_DEFAULT_REGION: "{{ aws_default_region }}"
|
|
|
|
- name: create systemd startup job for awsddns-debyl
|
|
include_tasks: podman/systemd-generate.yml
|
|
vars:
|
|
container_name: awsddns-debyl
|
|
|
|
# NOTE: git.debyl.io is an ALIAS record to home.debyl.io - no DDNS needed |