So after my last blog post I decided that this one should be less rushed, and more practised and tested, which turns out to be a good thing. After my last blog post the hosted servers I have didn't work over IPv6, this is due to the hosting firm's use of SLAAC to configure the external IPv6 address and routing, and my use of iptables to block all traffic that wasn't otherwise allowed. Now I allowed icmp echo requests on IPv4 but those commands raised an error run I transposed them to IPv6 so I left them out. This led to SLAAC, which requires ICMP to work over IPv6 to not work. That has been rectified now. So onto apache, and SSL certs. Now one of the requirements I had for these servers was the ability to swap between them via DNS, and as I do not know how to configure postfix to use multiple SSL certs based upon the domain that is being connected too I decided the easiest way to do that would be to get a cert with a cname to that shared domain for each server. Using http authentication with lets encrypt you put a file on disk and they request that file from the domain they are validating. This would be a problem for the server that is not currently being pointed at for the shared domain.
So how did I overcome this problem? Well I first installed Apache (as that is the web server I am most familiar with, and this could easily have been done with nginx, or any other webserver for that matter), opened up some additional ports (80, 443, and 8443 but only between each host not to the world) and set about configuring a loadbalancer pool that used a vhost bound to port 8443 on either server. That proved to be both ineffective and convoluted. With the loadbalancer features from mod_proxy_balancer I couldn't get the balancer to reliably failover in the event of getting a 404. I could get it to stop sending requests to that backend, but that didn't help the request that had already failed at that point, and it was also not necessarily desirable behaviour. After wrestling with this config for some time I decided to give up and get some sleep. It latter (the next day as it happens) dawned on me that what I wanted was a test to see if the file is on disk, and request the file from the server if it wasn't, this was straightforward with mod_rewrite instead of mod_proxy_balancer (mod_proxy was still needed mind). With this config if a file that doesn't exist is requested it causes a loop of requests that eventually grows too large and fails. This method could however be used to authenticate multiple servers just have them forwarding on requests in a loop, but as noted it does have a limit to how many servers you could use, unless you configured the servers to clean up the request headers, but that could open you up to a denial of service attack by way of getting apache to exhaust it's own connection pool with round robin requests that never end, so we'll avoid that for now.
So now I have ssl certs, and these can be automated (and will be once the servers are able to handle email) so next I have to configure postfix, and I intend to do so using virtual domains.