Random texts

freifunk

Got a message from a #freifunk colleague that users are unable to change their password on our mailserver. They just get bounced back into the login form of our PostfixAdmin after submitting it. Quick check: Yes, I have the same problem. Even the admin login is broken. No idea when it broke. #NixOS allows me to quickly activate an old configuration and software by executing a script (/nix/var/nix/profiles/system-476-link/bin/switch-to-configuration test), so I went back 15 days. That old generation worked. First success. Switching only takes a couple of seconds unless you care about kernel, etc. which would require a reboot. So finding the exact generation where it broke only took me about 5 minutes. But what causes it? I already had a guess, as I saw which services changed, but I wanted to be sure: nix-store -qR /nix/var/nix/profiles/system-476-link | sort -t- -k2 gives me the complete list of all included files and software in that configuration. So I dumped the known-good and known-bad lists and diff'ed them. /nix/store/...-dovecot-2.3.10.1 vs. /nix/store/...-dovecot-2.3.11.3 and a couple of unrelated libraries. PostfixAdmin or PHP did not change. But PostfixAdmin uses Dovecot to check passwords, e.g. during login. PostfixAdmin uses a simple command defined in the configuration file, so it should be easy to verify. Of course it works as root, but as the user that PostfixAdmin is actually running:

[pfa@mail:~]$ /nix/store/...-dovecot-2.3.10.1/bin/doveadm pw -r 12
doveadm(pfadmin): Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
Enter new password:
Retype new password:
{CRYPT}$2y$12$...

[pfa@mail:~]$ /nix/store/...-dovecot-2.3.11.3/bin/doveadm pw -r 12
doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 7: ssl_cert: Can't open file /var/lib/acme/mail.example.org/fullchain.pem: Permission denied

There's our culprit, Dovecot's new version breaks because it's unable to read a private key, which it doesn't even need for its current job. Apparently it's a known issue in Dovecot, as it has been reported on the Dovecot mailinglist about a week ago: https://dovecot.org/pipermail/dovecot/2020-August/119642.html There's even a workaround. Instead of specifying the ssl certificate in the config file, you move that part into a new config file that's only readable by root and use !include_try to include that file. Easy, right? Well, NixOS requires all config files to be world-readable (for users on that system). So I modified the dovecot service to create that root-only config file before starting. And PostfixAdmin is happy again and allows users to login and change their password.