On Friday, October 1st, 2021, we saw our SSL service failing to upload certificates to our hosting provider. We then stopped the service to investigate the cause of this.
To us, the certificates looked valid locally and had just been issued from Let’s Encrypt directly with no issues or warnings.
We then manually tried to upload the certificate to AWS and were met with a “certificate expired” error. We then reached out to AWS to confirm, and they confirmed the chain had an invalid certificate.
On September 30th, 2021, Let’s Encrypt’s root certificate (DST ROOT CA X3) expired, and while Let’s Encrypt had long ago added the new root certificate (ISRG Root X1) and tried to mitigate this issue by adding a cross-signed certificate that would point to both the new and old root certificates, it would not work with some versions of OpenSSL (<= v1.02) and it seems it was failing validation with AWS for simply containing an expired certificate, so while this was not affecting our current certificates, we had no way of uploading new certificates at this point.
After a lot of research, we had no option but to remove the expired certificate and the cross-signed intermediate certificates. This would then allow us to resume creating new certificates for our customers. This decision was not taken lightly, as while it means we can issue new certificates that work on 99.99% of modern devices, it would exclude older devices, like Android version 7.1 and older.
We do not take the responsibility our customers put on us lightly, and therefore we will continue working closely with our providers to see if we can resolve this for more devices, and to prevent this from happening in the future.
We appreciate your patience and continued trust in us 🙏