Graylog was the worst cost/benefit tenant on this 4-core box: two JVMs plus
MongoDB holding ~1.6 GB resident and ~3% CPU around the clock to store ~3k
messages a day -- about 28 MB across its four live indices. journald already
retains ~25 days of the same logs at its 500M cap, so this costs searchability,
not the logs.
The switch is `graylog_enabled` in inventory rather than a role default,
because three roles read it (common, podman, graylog-config). The disabled
path is an active teardown, not a skipped create: the containers already on
the host keep running and their systemd user units keep restarting them at
boot unless something stops and removes them. fluent-bit follows the same
flag -- with the GELF sink down it would spin retrying a dead 127.0.0.1:12202
and fill the journal it exists to drain -- but only the service state follows,
so re-enabling is a restart rather than a reinstall.
Caddy reloads were silently no-ops. The handler read /etc/caddy/Caddyfile,
which is a single-file bind mount, and podman binds those by inode; the
template module writes a temp file and renames it into place, so every deploy
gave the host file a new inode while the container kept seeing the one it was
created with. Config changes only ever landed when something recreated the
container. {{ caddy_path }}/config is also mounted, as a *directory*, and
directory mounts resolve names at open() time -- so /config/Caddyfile is
always the file Ansible just wrote.
awsddns and its four siblings had accumulated 12 zombies over 30 days of
uptime. The image's PID 1 is busybox crond, which only waitpid()s the job PIDs
it tracks and does no generic orphan reaping, so whenever the run-parts/sh
layer exited before the script it left a permanent <defunct>. init: true puts
catatonit at PID 1 to reap them, and the recreation clears the existing ones.
Also bumps fulfillr and greg-time-bot images.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add a second go-fulfillr container (fulfillr-dev) wired to the staging
Turso store + EasyPost/Stripe test keys via dev.json, served at
fulfillr-dev.debyltech.com (Caddy -> :9055), LAN-restricted like prod.
- fulfillr-dev.yml + dev.json.j2: the staging container, volumes, config
- defaults: fulfillr_dev_* vars; prod store URL stubbed off until cutover
- Caddyfile + caddy.yml: fulfillr-dev site block and static mount
- awsddns.yml: Route53 DDNS for the fulfillr-dev hostname
- production.json.j2: add store_database_url/store_auth, rename stripe key
var to fulfillr_stripe_api_key
- vault.yml: dev + store/stripe secrets
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add ollama role for local LLM inference (install, service, models)
- Add searxng container for private search
- Migrate hostname from home.bdebyl.net to home.debyl.io
(inventory, awsddns, zomboid entrypoint, home_server_name)
- Update vault with new secrets
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move fluent-bit to common role (systemd service, not a container)
- Move geoip to podman/tasks/data/ (data prep, not a container)
- Remove debyltech tag from geoip (not a debyltech service)
- Fix check_mode for fetch subuid task to enable dry-run mode
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit finalizes the comprehensive migration from nginx + ModSecurity + manual LetsEncrypt
to Caddy v2 with automatic HTTPS. The migration eliminates over 2000 lines of complex
configuration in favor of a single, simplified Caddyfile.
## Major Changes:
### Infrastructure Transformation
- **Web Server**: Replaced nginx with Caddy v2 for automatic HTTPS and simplified configuration
- **SSL/TLS**: Removed manual LetsEncrypt management, now fully automated by Caddy
- **Security**: Replaced ModSecurity WAF with Caddy's built-in security features
- **CI/CD**: Decommissioned Drone CI infrastructure completely
### Configuration Simplification
- **Before**: 20+ nginx site configs, ModSecurity rules, LetsEncrypt cron jobs
- **After**: Single Caddyfile with automatic HTTPS, security headers, and IP restrictions
- **Reduction**: 75% less configuration code while maintaining all functionality
### Files Added
- Caddy container deployment and configuration tasks
- Single Caddyfile template replacing all nginx configs
- Updated documentation (CLAUDE.md, TODO.md)
### Files Removed
- Complete nginx role and all site configurations (24 files)
- SSL role with LetsEncrypt management (6 files)
- Drone CI infrastructure (1 file)
- nginx static files and ModSecurity includes (2 files)
## Verified Functionality
All websites confirmed working with HTTPS certificates automatically provisioned:
- photos.bdebyl.net, parts.bdebyl.net, cloud.bdebyl.net
- wiki.skudakrennsport.com, cloud.skudakrennsport.com
- fulfillr.debyltech.com (with IP restrictions)
- Proper security headers and WebSocket support
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
