Automate Gateway and TLS Provisioning With Kong Gateway Operator and Cert Manager

(Updated: )
, , , , ,

In my previous post, we delved into utilizing the Kubernetes Gateway API. At that time, the Kong Gateway Operator was still in technical review, so we opted to manage the Kong deployment using Helm instead of the Gateway API.

Recently Kong has released version 1.2 of the Kong Gateway Operator, which they claim to be the “World’s First Open-Source Production-Ready Gateway API Operator.” In this post, I want to show you how to use the Kong Gateway Operator to manage Kong deployments. Additionally, we’ll take this opportunity to demonstrate how to use cert-manager to provision the Gateway API for TLS certificate management.

Let’s get started.

Installation

Kong Gateway Operator

We need to install Kong Gateway Operator first. It bundles Gateway API CRDs which is required for the next step.

  • Add helm repo
    1
    2
    helm repo add kong https://charts.konghq.com
    helm repo update kong
  • Here we specify the image tag to be 1.2.
    1
    2
    3
    4
    5
    helm upgrade \
      --install kgo kong/gateway-operator \
      -n kong-system \
      --create-namespace \
      --set image.tag=1.2

Cert-Manager

Next we will install Cert manager to handle our certificate needs.

If you are not familiar with cert-manager, please check out this post.

  • Add helm repo
    1
    helm repo add jetstack https://charts.jetstack.io
  • Here we need to enable a feature flag to supoprt Gateway API. Gateway API CRDs must be installed before.
    1
    2
    3
    4
    5
    6
    helm upgrade -i \
      cert-manager jetstack/cert-manager \
      --namespace cert-manager \
      --create-namespace \
      --set installCRDs=true \
      --set "extraArgs={--feature-gates=ExperimentalGatewayAPISupport=true}"

Create CA Issuer

I will create a CA issuer on my local cluster to create all certificates.

  • Create secret to store CA key pair
    1
    2
    3
    kubectl create secret tls \
      --namespace cert-manager ca-key-pair \
      --cert=ca.cert.pem --key=ca.key.pem
  • Create ClusterIssuser so this resource can be used from any namespace
    1
    2
    3
    4
    5
    6
    7
    apiVersion: cert-manager.io/v1
    kind: ClusterIssuer
    metadata:
      name: fomm-k8s-ca-issuer
    spec:
      ca:
        secretName: ca-key-pair

Gateway Deployment

In this demo I will deploy a shared Gateway in kong namespace for all resources to use.

1
kubectl create namespace kong

GatewayConfiguration

The GatewayConfiguration is used to customize a Gateway resource. Under the hook it creates ControlPlane and DataPlane when Gateway is requested.

You can still use Kong Gateawy Operator when you don’t want to use Gateway API. More detail can be found below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
kind: GatewayConfiguration
apiVersion: gateway-operator.konghq.com/v1beta1
metadata:
  name: kong
  namespace: kong
spec:
  dataPlaneOptions:
    network:
      services:
        ingress:
          type: LoadBalancer
          annotations:
            metallb.universe.tf/loadBalancerIPs: 192.168.18.150
    deployment:
      replicas: 1
      podTemplateSpec:
        spec:
          containers:
          - name: proxy
            image: kong:3.6
            env:
            - name: KONG_ROUTER_FLAVOR
              value: expressions
  controlPlaneOptions:
    deployment:
      podTemplateSpec:
        spec:
          containers:
          - name: controller
            image: kong/kubernetes-ingress-controller:3.1.3

This configuration is straightforward and easy to understand.

Control Plane (Kong Ingress Controller)
• Specify to use version 3.1.3.

Data Plane (Kong Proxy)
• Specify the version as 3.6.
• Utilize expressions router.
• Add an annotation to the LoadBalancer service to obtain an IP address of 192.168.18.150, allocated by MetalLB.

GatewayClass

The infrastructure provider has the flexibility to define various GatewayClasses for cluster operators to utilize. Each GatewayClass can be linked with distinct configuration settings or even different providers. In this demonstration, we will create a GatewayClass named kong that references the GatewayConfiguration we have just defined above.

1
2
3
4
5
6
7
8
9
10
11
kind: GatewayClass
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
  name: kong
spec:
  controllerName: konghq.com/gateway-operator
  parametersRef:
    group: gateway-operator.konghq.com
    kind: GatewayConfiguration
    name: kong
    namespace: kong

Gateway

The final step is to create our Gateway resource, which will be named kong-gw-demo. This gateway can then be referenced by name when defining an HttpRoute.

You may have noticed that we’ve added a cert-manager annotation to our Gateway resource. In contrast to traditional ingress configurations, where domain mappings are defined on an Ingress object, the Gateway API takes a different approach by delegating this responsibility to the Gateway itself. As such, we need to specify which domains can access this gateway and use cert-manager annotation to request certificates for the domains here. In this demo I’m requesting a certificate for proxy.li.k8s using the ClusterIssuer we created earlier. If you anticipate serving multiple domains with this Gateway, you can define additional listeners with distinct hostname. We’ve also used certificateRefs to specify the secret name where we’ll store our SSL certificates, which will be created in the same namespace as the Gateway.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
  name: kong-gw-demo
  namespace: kong
  annotations:
    cert-manager.io/cluster-issuer: fomm-k8s-ca-issuer
spec:
  gatewayClassName: kong
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All
    - name: https
      hostname: proxy.li.k8s
      port: 443
      protocol: HTTPS
      allowedRoutes:
        namespaces:
          from: All
      tls:
        mode: Terminate
        certificateRefs:
          - name: proxy-tls

Upon deploying the Gateway resource, you can expect to see two pods created as a result.

1
2
3
4
➜ kubectl get pods -n kong
NAME                                                     READY   STATUS    RESTARTS   AGE
controlplane-kong-gw-demo-bsfwh-m4j7f-698bb85f67-gmrbt   1/1     Running   0          3m29s
dataplane-kong-gw-demo-s2vb6-x4tdd-6b944748fc-d6lc4      1/1     Running   0          3m29s

Deploy KIC and Kong with CRD

Alternatively, if you are not ready to use the Gateway API, you can still benefit from using Kong gateway Operator to manage the deployment of Kong. To accomplish this, simply create ControlPlane and DataPlane resources in the same namespace.

ControlPlane

On the control plane, we need to make sure ingressClass is defined and specify the right data plane name.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
apiVersion: gateway-operator.konghq.com/v1beta1
kind: ControlPlane
metadata:
  name: controlplane-example
  namespace: kong
spec:
  ingressClass: kong
  dataplane: dataplane-example
  deployment:
    podTemplateSpec:
      spec:
        containers:
        - name: controller
          image: kong/kubernetes-ingress-controller:3.1.3

DataPlane

The DataPlane resource is the same as above.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: gateway-operator.konghq.com/v1beta1
kind: DataPlane
metadata:
  name: dataplane-example
  namespace: kong
spec:
  network:
    services:
      ingress:
        type: LoadBalancer
        annotations:
          metallb.universe.tf/loadBalancerIPs: 192.168.18.150
  deployment:
    replicas: 1
    podTemplateSpec:
      spec:
        containers:
        - name: proxy
          image: kong:3.5
          env:
          - name: KONG_ROUTER_FLAVOR
            value: expressions

Summary

In addition to using Helm, the Kong Gateway Operator is a valuable tool for managing your Kong deployment. It excels when you’re committed to leveraging the Gateway API and need to deploy multiple gateways, each responsible for managing ingress and egress for different namespaces or different listeners.

However, I believe there are several limitations to consider.

  • It only works when Kong is running in DBless mode.
  • if you only need a simple deployment as an entry point of your cluster, I’m not convinced it offers significant additional value compared to using Helm.
  • While the learning curve is not steep, you still need to familiarize yourself with new CRDs and the new CRDs does not provide much convenience and relies on podTemplateSpec heavily.
  • As the product is still in its early stages, some features are currently missing – for instance, the proxy service name is randomly generated and cannot be overridden.

Ultimately, you may find yourself deciding between using Helm or this tool. If you’re on board with the new Gateway API and don’t require Kong to run with a DB, which provides additional features like admin API and Kong Manager, KGO might be the better choice. If you need to deploy multiple Kong instances on your cluster, KGO is recommended. In all other cases, I think helm is a better choice at the moment.

That’s all I want to share with you today, see you on the next one.