Introduction to Kong Ingress Controller

(Updated: )
, ,

Intro

There are so many Ingress Controllers on the market and sometimes it is hard to choose. I’ve covered Traefik Ingress Controller in my previous post and today I will talk about using Kong Ingress Controller.

First of all, I would like to talk about how Kong ingress controller works. Unlike Nginx or Traefik Ingress Controller, you can see there are two containers in Kong pod, one is ingress-controller and the other is proxy. How this work is, when ingress-controller detects resources that is associated to Kong (through IngressClass or CRD), it will convert these resources to Kong objects and push it to proxy container. When we talk about Kong ingress controller, we are using Kong proxy at the same time. That means installing Kong and installing KIC on kubernetes are actually the same. The proxy container can work on its own, the KIC container howerver needs a proxy container to do the proxy part.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
📎 deployments(5)
├──🗂  nginx (1)
│  └──🪂 ingress-nginx-controller(1) [1/1/0]
│    └──🗂  default(1)
│       └──🚛 ingress-nginx-controller-fd7bb8d66-9m6pq(3) [1/1]
│          ├──🐳 controller
│          ├──💳 ingress-nginx(1) [automount=true]
│          │  └──🔒 ingress-nginx-token-t6cwv
│          └──🔒 ingress-nginx-admission
├──🗂  kong(1)
│  └──🪂 my-kong-kong(1) [1/1/0]
│     └──🗂  kong(1)
│        └──🚛 my-kong-kong-5d6b577bcd-dbmnw(3) [2/2]
│           ├──🐳 ingress-controller
│           ├──💳 my-kong-kong(1) [automount=true]
│           │  └──🔒 my-kong-kong-token-nv8lm
│           └──🐳 proxy
└──🗂  traefik(1)
   └──🪂 traefik(1) [1/1/0]
      └──🗂  traefik(1)
         └──🚛 traefik-8566cbc768-6vk5h(2) [1/1]
            ├──🐳 traefik
            └──💳 traefik(1)
               └──🔒 traefik-token-ndsdg

This information might not be important for a fresh installation, but you need to consider the changes to KIC and Kong when you need to upgrade.

I think that’s enough background information, let’s start using it. If you don’t know how to deploy Kong Ingress Controller on Kubernetes, please check my previous post.

You can find official docs below:

  • Official doc of annotations here
  • Official doc of CRDs here

Let’s get started.

Preparation

Add Kong helm repo

Before we start, we need to add Kong repo.

1
2
helm repo add kong https://charts.konghq.com
helm repo update

Install Kong

Let’s install Kong in DBless mode with below command first. This will create proxy service as LoadBalancer and I enable Admin API specifically.

1
2
3
4
5
6
helm upgrade -i my-kong kong/kong -n kong \
  --set image.tag=2.5 \
  --set admin.enabled=true \
  --set admin.http.enabled=true \
  --set ingressController.installCRDs=false \
  --create-namespace

Let me assign proxy.li.k8s to the IP address 172.18.18.150 of the proxy loadbalancer.

Similar to Traefik, you can use annotation or CRDs to config Kong. Howerver, annotations do not cover ALL features of CRD and Kong CRDs are used to extend Kubernetes native objects not to replace them.

If you have used Kong before, you would know you can use

  • Service object to set upstream.
  • Route objects to match requests and route to a service.
  • Consumer objects to be used to store credentials and rate limited when needed.
  • Plugins to do Magics.

This rules also apply when we use KIC.

To better understand how it works, let me give you a demo.

Deploy demo app

I will create a httpbin deployment for this demo.

  1. Create httpbin deployment. 
    1
    kubectl create deployment httpbin --image=kennethreitz/httpbin
  2. Expose this httpbin deployment as httpbin-svc service at port 80. 
    1
    kubectl expose deployment httpbin --name httpbin-svc --port 80
  3. Create Ingress and use Kong to route requests to this service. 
    1
    kubectl create ingress httpbin-route --class kong --rule '/test*=httpbin-svc:80'

Let’s visit our path and we should see below, which means it works.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
curl http://proxy.li.k8s/test -H "kong-debug:1" -i

HTTP/1.1 404 NOT FOUND
Content-Type: text/html; charset=UTF-8
Content-Length: 233
Connection: keep-alive
Kong-Route-Id: 2aa771dc-2f8e-5750-97b4-1d2da407764c
Kong-Route-Name: default.httpbin-route.00
Kong-Service-Id: 9b219a12-47ed-5405-b130-2beccaec4547
Kong-Service-Name: default.httpbin-svc.pnum-80
Server: gunicorn/19.9.0
Date: Wed, 22 Sep 2021 12:55:23 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
X-Kong-Upstream-Latency: 1
X-Kong-Proxy-Latency: 1
Via: kong/2.5.1

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>

How do I know it works? As you can see I passed in kong-debug:1 header in my request and I got service and route information in the response header. These headers confirms my request matched a Kong route.

1
2
3
4
Kong-Route-Id: 2aa771dc-2f8e-5750-97b4-1d2da407764c
Kong-Route-Name: default.httpbin-route.00
Kong-Service-Id: 9b219a12-47ed-5405-b130-2beccaec4547
Kong-Service-Name: default.httpbin-svc.pnum-80

Apparently we don’t want to see 404 from our app. So let’s try to extend our ingress object to make it work.

I will show you how to config Ingress or Service with annotations and KongIngress CRD. You can check the latest KongIngress CRD with below command:

1
2
3
kubectl kustomize \
  https://github.com/kong/kubernetes-ingress-controller/config/crd | \
  yq e '. | select(.spec.names.kind=="KongIngress")' -

Config on Ingress

Ingress Object will be converted to Kong route object once it is picked up by Kong ingress controller. To achieve the same Kong Route object features, we can use both annotation and KongIngress resource to extend Ingress.

Annotations on Ingress

konghq.com/strip-path

Let’s apply this annotation to our ingress first. This annotation will remove /test from our request sent to upstream.

1
kubectl annotate ingress httpbin-route konghq.com/strip-path=true

Let’s visit our route again, we should see HTTP/1.1 200 OK now because /test is striped, upstream receive our request at / hence it returns 200.

1
curl http://proxy.li.k8s/test -I

konghq.com/protocols

Don’t confuse this annotation with konghq.com/protocol for service resource.

1
kubectl annotate ingress httpbin-route konghq.com/protocols=https

This annotation is directly related to protocols in the route object which is about what protocol users want their client to send their request at. Because we only allow https through this annotation, you will see below if you access route via http.

1
2
3
4
5
6
7
8
9
10
11
12
13
curl http://proxy.li.k8s/test -i

HTTP/1.1 426
Date: Wed, 22 Sep 2021 13:14:54 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Upgrade: TLS/1.2, HTTP/1.1
Connection: Upgrade
Content-Length: 39
X-Kong-Response-Latency: 1
Server: kong/2.5.1

{"message":"Please use HTTPS protocol"}

By default Kong sends 426 response code for requests that need to be redirected to HTTPS.

konghq.com/https-redirect-status-code

If we want to send a redirect status code, we can use this annotation.

1
kubectl annotate ingress httpbin-route konghq.com/https-redirect-status-code=302

Now when we visit this route again, you can see we get a 302 response with a Location: https://proxy.li.k8s/test header. If you are using a browser it will redirect you to https automatically.

1
2
3
4
5
6
7
8
9
10
curl http://proxy.li.k8s/test -I

HTTP/1.1 302 Moved Temporarily
Date: Wed, 22 Sep 2021 13:20:07 GMT
Content-Type: text/html
Content-Length: 110
Connection: keep-alive
Location: https://proxy.li.k8s/test
X-Kong-Response-Latency: 1
Server: kong/2.5.1

konghq.com/preserve-host

This is used to preserve the Host header to upstream. As you can see, by default it is set to true

1
kubectl annotate ingress httpbin-route konghq.com/preserve-host=false

After we turn it off. We can see the Host header changed to an IP -> This is the IP of httpbin pod

1
2
3
4
5
6
7
8
curl https://proxy.li.k8s/test/anything -k
{
...
  "headers": {
    "Host": "10.244.0.7",
  },
...
}

konghq.com/methods

If we don’t set any methods, Kong accepts any request except TRACE.

Let’s limit this route to accept GET and POST method only.

1
kubectl annotate ingress httpbin-route konghq.com/methods=GET,POST

Now if we try PUT method, we will get route not match error because the method is a matching criteria now.

1
2
3
4
5
6
7
8
9
10
curl -XPUT https://proxy.li.k8s/test -ik

HTTP/2 404
date: Wed, 22 Sep 2021 14:01:21 GMT
content-type: application/json; charset=utf-8
content-length: 48
x-kong-response-latency: 0
server: kong/2.5.1

{"message":"no Route matched with those values"}

If we take a look at our Ingress Object now, it looks like below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    konghq.com/https-redirect-status-code: "302"
    konghq.com/methods: GET,POST
    konghq.com/preserve-host: "false"
    konghq.com/protocols: https
    konghq.com/strip-path: "true"
  name: httpbin-route
  namespace: default
spec:
  ingressClassName: kong
  rules:
  - http:
      paths:
      - backend:
          service:
            name: httpbin-svc
            port:
              number: 80
        path: /test
        pathType: Prefix

KongIngress on Ingress

We used 5 annotations on our ingress above. We can achieve the same configs with KongIngress object below.

  • Let’s start by removing existing Ingress object and create a new one

    1
    2
    kubectl delete ingress httpbin-route
    kubectl create ingress httpbin-route --class kong --rule '/test*=httpbin-svc:80'

  • Let’s save this file to kong-ingress-demo.yaml and then apply it kubectl apply -f kong-ingress-demo.yaml.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    apiVersion: configuration.konghq.com/v1
    kind: KongIngress
    metadata:
      name: kong-ingress-demo
      namespace: default
    route:
      methods:
      - POST
      - GET
      strip_path: true
      preserve_host: false
      https_redirect_status_code: 302
      protocols:
      - https

  • Let’s associate this KongIngress to our ingress

    1
    kubectl annotate ingress httpbin-route konghq.com/override=kong-ingress-demo

Now we have the same config as using 5 annotation above. If we need to apply the same config to another ingress object, we just need to associate the same KongIngress resource to the new ingress object.

Unique Features

There are features that are only available with annotation or KongIngress. I will show you a couple below.

konghq.com/host-aliases

You can add multiple hostnames as a matching criterial on the same ingress object. Let’s add annotation to our ingress.

1
kubectl annotate ingress httpbin-route konghq.com/host-aliases=foo.li.k8s,bar.li.k8s

Now when we vist curl https://proxy.li.k8s/test -Ik again, we will get 404 not found because our request does not match any route objects. If we send our request with foo.li.k8s or bar.li.k8s host header, we will see 200 response again.

1
2
3
curl https://proxy.li.k8s/test -Ik -H "Host: foo.li.k8s"

HTTP/2 200

This is handy when you need to match matching multiple Host values to the same set of routes.

Match Headers with KongIngress

If you need direct traffics to different back ends for different headers, you can use KongIngress to do it. Let’s extend our KongIngress a little bit.

This will add host header demo-header-1 to match value demo1 or demo2 and header demo-header-2 to match value demo3.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
  name: kong-ingress-demo
  namespace: default
route:
  methods:
  - POST
  - GET
  strip_path: true
  preserve_host: false
  https_redirect_status_code: 302
  protocols:
  - https
  headers:
    demo-header-1:
    - demo1
    - demo2
    demo-header-2:
    - demo3

Let’s visit our route again.

We will be getting no Route matched error now.

1
2
3
curl https://proxy.li.k8s/test -iLk

{"message":"no Route matched with those values"}

Let’s try adding some headers to our request

1
2
3
curl https://proxy.li.k8s/test -iLk \
  -H "demo-header-1:demo1" \
  -H "demo-header-2:demo3"

We should see HTTP/2 200 again.

Config on service

You can also extend service with both annotations and KongIngress object.

Same KongIngress object can be applied on both Ingress and Service object with different features.

Annotations on Service

konghq.com/protocol

Don’t confuse this with konghq.com/protocols annotation on ingress resources, You can use this annotation to define what protocol Kong use to communicate with upstream services. Accepted values are http,https,grpc,grpcs,tcp,tls.

1
kubectl annotate service httpbin-svc konghq.com/protocol=http

konghq.com/path

This annotation can be used to prepend an HTTP path of a request before the request is forwarded.

1
kubectl annotate service httpbin-svc konghq.com/path=/anything

If we visit our route at /test again

1
curl http://proxy.li.k8s/test -i

As you can see the upstream has the path /anything prepended.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
{
  "args": {},
  "data": "",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Connection": "keep-alive",
    "Host": "proxy.li.k8s",
    "User-Agent": "curl/7.74.0",
    "X-Forwarded-Host": "proxy.li.k8s",
    "X-Forwarded-Path": "/test",
    "X-Forwarded-Prefix": "/test"
  },
  "json": null,
  "method": "GET",
  "origin": "10.244.0.1",
  "url": "http://proxy.li.k8s/anything"
}

KongIngress on Service

We can also achieve the same feature as above annotations with KongIngress object.

  • To remove the annocations quickly, let’s remove and re-create the same service.

    1
    2
    kubectl delete service httpbin-svc
    kubectl expose deployment httpbin --name httpbin-svc --port 80

  • Then we need to create our KongIngress resource.

    1
    2
    3
    4
    5
    6
    7
    8
    apiVersion: configuration.konghq.com/v1
    kind: KongIngress
    metadata:
      name: kong-service-demo
      namespace: default
    proxy:
      protocol: http
      path: /anything

    Save it to kong-service-demo.yaml and then apply kubectl apply -f kong-service-demo.yaml.

  • Lastly let’s associate this KongIngress resource to our service

    1
    kubectl annotate service httpbin-svc konghq.com/override=kong-service-demo

After that we can visit /test route again and we should see the same result as above.

Unique Features

There are more unique features that is only available with KongIngress for service.

konghq.com/client-cert

This annotation is used to provide a client-certificate to upstream server if it requires mutual authentication.

We need to make sure either use konghq.com/protocol annotation or KongIngress to communicate upstream service with https protocol only.

We need to create the client certificate as secret first. My client certificate is stored in example-client-cert in default namespace, same as my ingress.

1
kubectl create secret tls example-client-cert --cert=example.cert.pem --key=example.key.pem

Then we just apply this annotation to our service.

1
kubectl annotate service httpbin-svc konghq.com/client-cert=example-client-cert

We can port forward admin api and check the service object. We should see something similar on it.

1
2
3
"client_certificate": {
  "id": "e31e7492-7337-4a42-a386-3369039f7cbf"
}

Health Check

You can use KongIngress to change the behaviour of an upstream object or service object. One unique feature is health check and circuit breaks on upstream object. We can also config healthcheck with KongIngress CRD.

Let’s modify our KongIngress resource to below. This will instruct Kong to do active health check to upstream service at /health endpoint every second. If the health check at one target fail 5 times, this target will be marked as unhealthy and Kong will not send request to it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
  name: kong-service-demo
  namespace: default
proxy:
  protocol: http
  path: /anything
upstream:
  healthchecks:
    active:
      healthy:
        interval: 1
        http_statuses:
        - 200
        - 302
      http_path: /health
      unhealthy:
        http_failures: 5
        http_statuses:
        - 429
        - 404
        - 500
        - 501
        - 502
        - 503
        - 504
        - 505

That’s all I want to share in this post. The other CRDs like KongPlugin, KongClusterPlugin and KongConsumers are hard to wirte one post to cover them all. I will cover those CRDs in my plugin posts like I did before.

Thanks for reading, see you next time.