Bastian de BylandClaude Opus 5
f21be79452
Three findings from investigating sustained I/O pressure on the root SSD.
Grouped because the logging and storage edits land in the same task file.
rsyslog was writing a second complete copy of the journal to
/var/log/messages: 6.7 GB of rotated copies, two weekly files of which were
2.8 GB and 2.5 GB. It loads imjournal, so it reads the journal directly and
ForwardToSyslog=no alone does not stop it -- the unit itself has to go.
Verified nothing consumes those files first: fail2ban runs backend=systemd and
matches on the journal ("No file is currently monitored"), and lsof showed only
rsyslogd holding them. Measured afterwards: writes 46 -> 23 GB/day, /var/log
6.7 GB -> 655 MB, journald still capturing container stdout.
The SSD was on bfq, which fedora's stock 60-block-scheduler.rules picks for any
rotational=0 disk. bfq is built for spinning disks and desktop interactivity:
it costs CPU per request, lets reads queue behind write bursts, and hard-caps
nr_requests at 64. With 26 containers and two CI runners writing at once that
is the wrong trade. mq-deadline rather than none because this is SATA with a
32-deep NCQ queue, not NVMe -- the merging and the read-expiry deadline both
earn their place.
Writeback was at the stock percent-of-RAM ratios, so on 31 GB the kernel would
sit on 3.1 GB before starting writeback and 6.2 GB before blocking writers.
Flushing that to a QLC drive that falls to ~80-160 MB/s once its SLC cache is
spent takes tens of seconds with everything stalled behind it. Capped in
absolute bytes instead: one long stall traded for frequent short ones.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deploy Home
There's no place like home!
Just as Dorothy managed the simple task of clicking her heels together, the desire for an equally simple one-button push deployment was in my heart. Thus, this repository was made.
Ansible
Ansible, along with double encrypted secrets, deploys the necessary configurations to make the home fit for certain needs and desires. Namely, having access to my home from anywhere, securely, and a self-hosted CI server that easily ties into existing workflows.
Makefile
The makefile is primarily used as a wrapper script to ensure that necessary
files, such as the secret vault password file, are provisioned as part of this.
One such addition to the task is utilizing dependency pinning through the
utilization of Python's virtualenv to lock down the specific dependency
versions within the requirements.txt file. This, ideally, prevents any
deployment issues with dependency version woes (e.g. version conflicts, major
updates in newest versions, etc.)
| Target Name | Description |
|---|---|
lint |
(default) Runs yamllint and ansible-lint on all YAML files in ansible/ |
deploy |
Deploys everything, or only tasks specified in TAGS= environment variable |
check |
Runs deploy in a "dry-run", showing diff-style outputs on tasks indicating changes |
vault |
Opens the Ansible vault file for editing |