A Complete Guide of SSL and the Provisioning of It With Traefik on Docker

(Updated: )
, ,

Traefik Labs released version 2.4.0-rc1 recently. This version includes an ACME enhancement called External Account Binding which I had been waiting for quite some time now. After doing some testing, I can finally start writing this post. This post is aiming to provide a comprehensive review of using TLS certificates with Traefik. Although the environment I used was Docker, you should apply the same rules on other platforms.

TL; DR

It is very easy to provision TLS certificates to your server automatically with Traefik. What you need to do is to define a certResolver in your static configurations and then tell Traefik how you want to use the certificate. Traefik will do the rest for you.

An example of certResover :

1
2
3
4
5
6
7
8
9
10
# Static configuration
certificatesResolvers:
  <NAME_YOUR_PROVIDER>:
    acme:
      caServer: <CA_ACME_DIRECTORY>
      email: <YOUR_VALID_EMAIL>
      storage: acme.json
      <EXTERNAL_ACCOUNT_BINDING>
      keyType: <PREFERRED_KEY_ALGORITHM>
      <VALIDATION_METHOD>

  • NAME_YOUR_PROVIDER

    You can give your certResolver any name. For example, LE-http to indicate it is a provider for LetsEncrypt with http-01 validation method. You can create multiple certificatesResolvers and choose the right one for your router.

  • YOUR_VALID_EMAIL :

    Provide a valid email address to recieve renewal notification. Some CA will reject your requests if you use bogus email addresses.

  • CA_ACME_DIRECTORY

    Some commercial CAs does not have a fixed ACME URL. Users need to generate ACME directory URL from their accounts. Please follow your certificate provider’s instructions to generate these urls.

  • PREFERRED_KEY_ALGORITHM

    Although CAB forum allows the use of 521 bit ECC key, most CAs only accept 256 or 384 bits ECC keys.

  • VALIDATION_METHOD

  • EXTERNAL_ACCOUNT_BINDING (EAB)

    Commercial CAs normally require users to generate EAB credentials from their accounts to pair with their ACME URLs. Although Zerossl is free, you still need to create an account and genreate EAB credentials as it is under Sectigo’s root.

SSL Basics

The whole PKI industry had been forced to adapt some critical changes In the past few years. Major browsers like Chrome started marking http webpages not secured since 2018, certificate validty was reduced from 3 years -> 2 years and it is reduced to 1 year. Especially after the official release of Let’s Encrypt, SSL certificate should be default feature of any public facing websites. I won’t be covering what SSL is or how SSL worksto day, I would be talking about a few SSL concepts and then explain the configurations I use with Traefik.

Key Algorithm

The CAB forum baseline requirement currently requires key strengths of all issued TLS certificates are at least 2048-bit RSA using SHA-256, SHA-384 or SHA-512 or Elliptic Curve using NIST P-256, P-384, or P-521. Most of the TLS certificate issued today are based on RSA 2048 keys. What is ECC key?

from Wikipedia:

Elliptic-curve cryptography (ECC) is an approach to public-key cryptography based on the algebraic structure of elliptic curves over finite fields. ECC allows smaller keys compared to non-EC cryptography (based on plain Galois fields) to provide equivalent security

ECC Key Size RSA Key Size
256-bit 3072-bit
384-bit 7680-bit
521-bit 15360-bit

According to this article, ECC certificates has smaller size and better security and performance. Unless very old systems/devices support are need, I reckon everyone should start using ECC private keys to generate their TLS certificates.

Certificate coverage

Except EV certificate, users can get a single domain, a multi-domain certificate or a wildcard certificates depends on their needs. Single/Multi-Domain SSL are pretty self-explanatory, your certificate covers a single or multiple FQDN. Wildcard certificate on the other hand covers ALL sub-domains of one level. For example, *.my.domain covers <EVERYTHING>.my.domain. As there is nothing in the front, my.domain is not covered. If you want it to be covered, you need to add it as SAN to your wildcard certificate.

Why do we use different CAs?

There are 5 aspects users need to consider from which Certificate authority they get their certificates from.

Reputation

This option looks stupid, why would anyone get their cert from a non-reputatble CA? When you are applying a certificate you naturally want to make sure it will work till it expired. Choosing certificates from a more reputatble CA might help users avoid service distruption if certificate needs to be revoked and replaced in time. Let’sEncrypt revoked 3 million certificates in March 2020 due to a CAA check bug they had . You can read the whole story here.

The root matters

TLS certificates are trusted because their roots are pre-installed and trusted on softwares, devices, browsers or operating systems. It is critical to know if certificate’s root is supported on clients’ environment before making the purchase. SSL certificate won’t be trusted if its root is not pre-installed on clients’ devices. For example, . A good example is again, Let’sEncrypt, they are cross signing their certificates with ISRG Root X1(newer root) and DST Root CA X3 roots. If most of your clients are using older systems, you probably want to avoid using Let’sEncrypt’s newer root to sign your certificate.

Clarity of workflow

Users need to know what the steps are to get their certificates issued. Even for a simple DV certificate, sometiems it takes a long time simply because users are unclear about what they need to do to get their certificates issued.

Validation Method

Although CAB forum baseline requirement defines how validation should be done, different CAs have their own interpretations. It is important to get your certificate from a CA that provides easiest validation methods. For example, DNS validation, some CAs only allow DNS CNAME records while the others allow both DNS TXT or DNS CNAME records.

OCSP response speed

This is often overlooked by users but it is actually critical. Every time your client connects to your server via https, there will be some sort of revocation checks behind the scene. Chrome is using CRLset, Firefox and others are using OCSP to check certificate status. (This is part of the reason people feel Chrome is faster than its rivals) There are lots of software and devices are using OCSP to do certificate checks. It is very important to make sure your certificate OCSP response is quick and stable. If you want to know more about how SSL revocation check affects connection speed, you can read this article.

Let’s move on to today’s main topic.

SSL certificate provisioning

You might have already sensed that, I take TLS security seriously. IMO no public facing application/webpages should be deployed without TLS. Ever since I found Traefik, it quickly became my MUST-HAVE tool to deploy applications on Docker. I can hide all application at the back and use Traefik to handle all SSL handshakes without configuring each application itself.

Traditional Method

The traditional way of provisioning TLS certificate can be summarised in 5 steps.

  1. Generate private key and CSR on users server/device
  2. Provide CSR to the CA.
  3. Complete required validation.
  4. CA issue the certificate to you in the format your server needs.
  5. Install this certificate on your server.

If you prefer using TLS certificates this way with Traefik, you can check my previous post for more detail.

Automated Method with Traefik

If you are lazy as I am, you probably want your tool to do as much work as possible for you. Provisioning TLS certificate via ACME protocol does exactly that. Tools like certbot and cert-manger have been widely used for quite some time now. Traefik also utilise ACME protocol for provisioning certificates.

Most commercial CAs should support ACME protocols nowadays. If your CA does not support ACME, you probably need to consider purchasing certificates somewhere else. (IMO there is no excuse for a commercial CA not to have ACME support)

Free SSL providers

Here are 3 free SSL certificate providers that issue certificates free of charge to everyone via ACME protocol.

CA Name Rate Limit FQDN Limit preferredChain Wildcard Required EAB
Let’s Encrypt 50/week 100 Names/cert Yes Yes No
Buypass 20/week 5 Name/cert No No No
ZeroSSL No No No Yes Yes

You can find their root certificates below.

ZeroSSL’s intermediate certificates are issued under Sectigo’s root. Hence the roots name contain USERTrust.

A fun fact: A lot of certificate providers are merely sub-CAs, they do NOT have their own roots. I was surprised to find out Sectigo (formerly Comodo) had issued a HUGE number of sub-CAs. You can find all sub-CAs certificates issued under these two USERTrust roots here if you are interested.

Name Valid From Valid To Signature Algorithm Serial
DST Root CA X3 2000-09-30 2021-09-30 SHA1-RSA 44afb080d6a327ba893039862ef8406b
ISRG Root X1 2015-06-04 2035-06-04 SHA256-RSA 8210cfb0d240e3594463e0bb63828b00
Buypass Class 2 Root CA 2010-10-26 2040-10-26 SHA256-RSA 2
USERTrust ECC Certification Authority 2010-02-01 2038-01-19 ECDSA-SHA384 5c8b99c55a94c5d27156decd8980cc26
USERTrust RSA Certification Authority 2010-02-01 2038-01-19 SHA384-RSA 01fd6d30fca3ca51a81bbc640e35032d

CA ACME directory URLs

Free CA production ACME directory URLs.

  • Let’sEncrypt

    1
    https://acme-v02.api.letsencrypt.org/directory

  • Buypass

    1
    https://api.buypass.com/acme/directory

  • ZeroSSL

    1
    https://acme.zerossl.com/v2/DV90

Validation Methods

You need to define validation method of your certResolver. There are three methods, http-01, dns-01 and tls-01. I will only cover the first two.

DNS-01
1
2
3
4
5
dnsChallenge:
  provider: <DNS_PROVIDER_NAME>
  resolvers:
    - "1.1.1.1:53"
    - "1.0.0.1:53"

**DNS_PROVIDER_NAME **: Please provide your dns provider here. You can check the full list of supported providers here.

You can also use custom DNS servers to resolve FQDN by adding resolvers param. In above example, I am using cloudflare as my DNS resolver because it is the fastest resolver according to dnsperf.

HTTP-01

You just need to let Traefik know the name of your port 80 entry point. In official documentation, port 80 entry point name is web. 

1
2
3
4
5
6
7
8
entryPoints:
  web:
    address: ":80"
    
			...
			httpChallenge:
        entryPoint: web
      ...

Preferred chain

If you feel the need of using different root with LetsEncrypt after reading above section, you can add preferredChain param to your ACME resolver. You can find official doc here.

1
preferredChain: 'DST Root CA X3'

External Account Binding

It is actually pretty straight forward, you just need to add below to your ACME resolver. You can find the official documentation here.

1
2
3
eab:
  kid: abc-keyID-xyz
  hmacEncoded: abc-hmac-xyz

Usage Examples

After reading above, I hope you will have some ideas of what certificate you want to use for your service. Let me list down some examples:

To define providers

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
entryPoints:
  web:
    address: ":80"

certificatesResolvers:
  letsencrypt:
    acme:
      email: [email protected]
      storage: acme.json
      keyType: EC384
      httpChallenge:
        entryPoint: web
        
  letsencrypt-legacy:
    acme:
      email: [email protected]
      storage: acme.json
      keyType: EC384
      preferredChain: 'DST Root CA X3'
      httpChallenge:
        entryPoint: web    
        
  buypass:
    acme:
      email: [email protected]
      storage: acme.json
      caServer: https://api.buypass.com/acme/directory 
      keyType: EC256
      httpChallenge:
        entryPoint: web
        
  le-dns:
    acme:
      email: [email protected]
      storage: acme.json
      keyType: EC384
      dnsChallenge:
        provider: acme-dns
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"
          
  zerossl:
    acme:
      email: [email protected]
      storage: acme.json
      caServer: https://acme.zerossl.com/v2/DV90
      keyType: EC384
      eab:
        kid: <KEY_ID>
        hmacEncoded: <HMAC_VALUE>
      dnsChallenge:
        provider: cloudflare
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

As you can see, the first two are using http-01 validation method with entry point web. If you choose to use Let’sEncrypt, you don’t need to specify caServer url. You might have also noticed that I am using the same email address for all certResolver, that’s because of a bug (I believe it is a bug) that was briefly mentioned here. Frankly this requirement makes no sense to me…🥱

The next two providers are using dns-01. One is using acme-dns provider, the other one is using cloudflare. acme-dns is useful if your DNS management provider is not a supported provider. To know more about using acme-dns, pleaes read this post.

To use Providers

Official configuration examples shows how to assign certResolver to each router.

Defining a certificates resolver does not result in all routers automatically using it. Each router that is supposed to use the resolver must reference it.

You can also define a default certResolvrer in your static configuration to avoid adding cert resolver to each router. My static config does exactly that. As you can see from below code snippet, I am using letsencypt as my default resolver.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
entryPoints:
...
  websecure:
    address: :443
    http:
      middlewares:
        - secureHeaders@file
      tls:
        certResolver: letsencrypt

certificatesResolvers:
  letsencrypt:
    acme:
      email: admin@yourdomain
      storage: acme.json
      keyType: EC384
      httpChallenge:
        entryPoint: web
...

Extra Settings

  • entryPoint http redirect to https

    This feature was introduced in v2.2. I was so excited about its release I even wrote a post for this feature. You can find official documentation here.

  • HSTS secure headers and Mozilla SSL Intermediate config

    If you followed any of my tutorials before, you may have noticed that I’ve been using these setting in my dynamic configuration.

    • I define a headers middleware called secureHeaders. This is for HSTS. Please check please check Wikipedia or Chromium HSTS preload list website to understand each setting and make your own decision if you want to use it or not, especially stsPreload.

      Preloading Should Be Opt-In

      If you maintain a project that provides HTTPS configuration advice or provides an option to enable HSTS, do not include the preload directive by default. We get regular emails from site operators who tried out HSTS this way, only to find themselves on the preload list by the time they find they need to remove HSTS to access certain subdomains. Removal tends to be slow and painful for those sites.

    • I got my TLS options from Mozilla SSL Configuration Generator. I chose the intermediate config for better compatibility.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    # Dynamic configuration
    http:
      middlewares:
        secureHeaders:
          headers:
            sslRedirect: true
            forceSTSHeader: true
            stsIncludeSubdomains: true
            stsPreload: true
            stsSeconds: 31536000   
    tls:
      options:
        default:
          cipherSuites:
            - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
            - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
            - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
            - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
            - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
            - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
          minVersion: VersionTLS12

That’s all I want to share today. I did not expect to write such a long post before I started but I guess it is a really good start of 2021.

Thanks for reading, see you next time.