Fix Broken LetsEncrypt SSL Certificate due to Expired Root CA Certificate

How can we help?
< All Topics

Fix Broken LetsEncrypt SSL Certificate due to Expired Root CA Certificate

After 30th September 2021, if you are owning a website or service that uses LetsEncrypt as its SSL Certificate issuer, your website may throw a certificate error when accessed. You are then taking a look at the expired date of the SSL Certificate and found actually still have time and it should not be throwing an expired error.

R3 Intermediate certificate has expired

What is actually happening?

As we can see above, the Device certificate itself is not yet expired. However, there’s a chain above the device certificate that has already expired. That is the Certificate Authority (CA) certificate that is being used as a root chain of the LetsEncrypt certificate.

Root Chain? Certificate Authority? What is that?

That is a term used in Public Key Infrastructure (PKI), which will become another discussion. However, if you would like more detailed info, you can read the Wikipedia article on PKI (Public key infrastructure – Wikipedia) and this video published by Securemetric which explained what PKI is in a more detailed manner is (What is Public Key Infrastructure (PKI) by Securemetric – YouTube).

In simpler terms, all certificates that are used to deploy SSL on the Web are issued by a Certificate Authority, a trusted organisation recognised by your device/OS in the form of another SSL certificate. Every OS and device have their own set of preinstalled trusted CA Certificates.

Running below code in Ubuntu will list the trusted CA Certificates installed in the system:

awk -v cmd='openssl x509 -noout -subject' '
    /BEGIN/{close(cmd)};{print | cmd}' < /etc/ssl/certs/ca-certificates.crt

Or if you are using Windows, you can open Certificates Snap-in in mmc.exe and you will see Installed Trusted CA Certificates

Same with device certificates, CA certificates have their own Expiration date. Even though they are usually longer (5-25 years validity)

On 30th September 2021, DST Root CA X3, which is the CA Certificate used by Let’s Encrypt, is expired. This is the source of the problem.

Let’s Encrypt had planned to move away from the DST CA root to their own root, ISRG Root X1, that expires on 4th June 2035. The problem was, only a few devices had received the necessary updates that include this new ISRG Root X1. If a vast selection of devices had not received an update to include this new root certificate, they simply won’t trust it.

The solution taken by Let’s Encrypt was a cross-signing certificate. Until DST CA X3 expire, they deploy certificates with a cross-sign. Visual representation for the chain issuance below

With this model, however, after IdenTrust DST Root CA X3 expires, some old devices will not trust Let’s Encrypt anymore. Based on Lets Encrypt Compatibility List, Certificate will not work on devices / OSes below:

  • macOS < 10.12.1
  • iOS < 10
  • Mozilla Firefox < 50
  • Ubuntu >= precise / 12.04 and < xenial / 16.04
  • Debian >= squeeze / 6 and < jessie /8
  • NSS >= v3.11.9 and < 3.26
  • Amazon FireOS (Silk Browser) (version range unknown)
  • Cyanogen > v10 (version that added ISRG Root X1 unknown)
  • Jolla Sailfish OS > v1.1.2.16 (version that added ISRG Root X1 unknown)
  • Kindle > v3.4.1 (version that added ISRG Root X1 unknown)
  • Blackberry >= 10.3.3 (version that added ISRG Root X1 unknown)
  • PS4 game console with firmware >= 5.00 (version that added ISRG Root X1 unknown)
  • Blackberry < v10.3.3
  • Android < v2.3.6
  • Nintendo 3DS
  • Windows XP prior to SP3cannot handle SHA-2 signed certificates
  • Java 7 < 7u111
  • Java 8 < 8u101
  • Windows Live Mail (2012 mail client, not webmail)cannot handle certificates without a CRL
  • PS3 game console
  • PS4 game console with firmware < 5.00

Supported devices still having issue

On 30th September 2021, However, some supported OSes and devices still have errors with the Root certificate.

This is due to some devices still reading the Certificate chain root back to DST X3 and not ISRG X1. Even though a website is having a cross-signed. Thus, makes the system believe the root certificate is expired.
If we checked using the OpenSSL command, the header for the CA certificate would look like this

This will eventually be replaced by Let’s Encrypt. However, this issue will create more problems. We need to “force” to replace the Certificate.

Fixing the Chain

The first way to fix the chain is to “force renew” the device certificate.

If you are using certbot to deploy and renew the certificate, you can run the following code to “force renew”:

certbot run -a webroot -i <your webserver application> -d <domain name> --force-renewal

certbot run -a webroot -i nginx -d --force-renewal

Reboot your server and wait for a few moments.

Check again using the OpenSSL command, if root certificate has the following header, it means that it has been successfully replaced.

If not, you need to change the deployed certificate manually.

First, you need to download the ISRG Root X1 pem and copy the valid ISRG Root X1 certificate from Let’s Encrypt to a text file. Then open a notepad or another text editor. You will find the following content inside the .pem file:


Then, open your web server via SSH, and navigate to the folder path in which you save the certificate chain file. If you are using certbot / Ubuntu, you should find the certificate in /etc/letsencrypt/live/ (change to your own domain name). In this folder, there will be four .pem files. Open fullchain.pem using a text editor.

When you open the fullchain.pem certificate, you will notice that the file will be structured like following:

Base-64 Encoded Text #1
Base-64 Encoded Text #2
Base-64 Encoded Text #3

This means that this fullchain.pem consists of three different certificates. Device Certificate, Intermediate Certificate, and Root Certificate. Since you only need to change the Root Certificate, you will need to replace the last Base-64 Encoded Text with the certificate above. After you replace it, save the file, and then open chain.pem.

Just like a before, chain.pem consists of multiple certificates. However, it is only containing intermediate, and root certificates. you will need to replace the last Base-64 Encoded Text with the certificate above. After you replaced it, save the file. and restart the Web Server.

After a moment, you will see that your website is accessible, and if you check the certificate, it will use ISRG Root X1 as its root


Root certificate expiration really creates an impact on the internet. Big names and companies are also using it for their services. However, Let’s Encrypt also have their own solution for system administrators.

Find more practical article in Cloud Raya’s Knowledge base. Or better, you can sign up to Cloud Raya’s to find what you can do from our simple and multifunctional dashboard.


Table of Contents

Comments are closed.

Ready, Set, Cloud

Ready, Set, Cloud