Explore Kubernetes Gateway API With Kong Ingress Controller

Kubernetes has become the standard for orchestrating containerized applications, offering declarative and automated management. Traditionally, ingress objects have been employed to handle external traffic to micro services within the cluster. However, to make Ingress resources effective, an ingress controller must be operational in the cluster. Despite its widespread use, Ingress comes with limitations, such as exclusively managing L7 traffic, restricting routing to services within the same namespace, and lacking features like header and query string matching.

To address these limitations, Gateway API has been introduced. It exposes a more versatile proxy API that supports multiple protocols beyond HTTP and models various infrastructure components. This offers enhanced deployment and management options for cluster operations. Roles and personas also play a crucial role in Gateway API, delineating distinct responsibilities for different teams within the system.

In this post, I’ll explore the standard version of Gateway API and showcase how you can utilize the Kong Ingress Controller to understand the capabilities offered by Gateway API.

Install Gateway API CRD

The Gateway API has certain CRDs that require manual installation. As previously mentioned, I will install the CRDs from the standard channel and explore Package v1 which includes:

• Gateway
• GatewayClass
• HTTPRoute

kubectl apply -f \
  https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml

Infrastructure Ops

With the CRDs installed, the next step is to define GatewayClass objects, which describe different types of Gateway instances. The official documentation can be a bit confusing, especially when it mentions, "Note: GatewayClass serves the same function as the networking.IngressClass resource." In reality, GatewayClass defines how you want to deploy your Gateway instances with different configurations.

Infrastructure providers can offer both internal and external GatewayClass options, enabling distinct features for different gateway instances. For instance, for an internal gateway, you may want to enable stream listening, while for an external gateway, you would only enable proxy listening.

Create GatewayClass

Let’s create the GatewayClass kong-demo in our cluster and set the controller name to be kic-gateway-controller. This is the default value for Kong Ingress Controller to reconciles any resources attached to this GatewayClass. For additional information, please refer to Kong official doc.

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: kong-demo
  annotations:
    konghq.com/gatewayclass-unmanaged: 'true'
spec:
  controllerName: konghq.com/kic-gateway-controller

Cluster Operations

Gateway objects are meant to spin up gateway instances dynamically based on the GateayClass it reference to and the listeners they define. However, the Kong Gateway Operator is still in tech preview, we will deploy Kong manually.

Deploy Kong Ingress Controller

We will use helm to install, let’s add Kong repo and update first.

helm repo add kong https://charts.konghq.com
helm repo update

We need to make sure the kong/ingress chart version is 0.10.2 or later.

helm search repo kong/ingress -o json | jq .[].version

Next we can install Kong Ingress Controller with release name my-kong.

helm upgrade -i my-kong kong/ingress -n kong \
  --set gateway.image.tag=3.5 \
  --set gateway.env.router_flavor=expressions \
  --set gateway.admin.enabled=true \
  --set gateway.admin.http.enabled=true \
  --set controller.ingressController.installCRDs=false \
  --create-namespace

If you use MetalLB and it does not assign LoadBalancer IP automatically, you can annotate your proxy service <helm_release_name>-kong-proxy with the desired IP as shown below.

kubectl -n kong annotate \
  service my-kong-gateway-proxy \
  metallb.universe.tf/loadBalancerIPs=192.168.18.150

Create Gateway

As we are deploying Kong Ingress Controller manually, it’s crucial to ensure that the listener on the Gateway object matches the Kong Proxy service. You can create the Gateway in the same namespace as your HTTPRoute. If you want to use the same Gateway to route traffic across the cluster, define a Gateway in one namespace and specifically allow HTTPRoute to use it. In the example below, I create the Gateway kong-gw-demo in the kong namespace. Then, I allow HTTPRoute from namespaces with the label shared-gateway-access: "true" to use this Gateway.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong-gw-demo
  namespace: kong
spec:
  gatewayClassName: kong-demo
  listeners:
    - name: proxy
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              shared-gateway-access: "true"

If you want to limit the usage of Gateway to specific namespaces without extra label, you can define your gateway as below.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong-gw-demo
  namespace: kong
spec:
  gatewayClassName: kong-demo
  listeners:
    - name: proxy
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchExpressions:
              - key: kubernetes.io/metadata.name
                operator: In
                values:
                  - httpbin
                  - nginx

Create demo Apps

Next we will deploy two applications for our demos.

kubectl create namespace httpbin
kubectl label namespace httpbin shared-gateway-access=true
kubectl -n httpbin create deployment httpbin --image=mccutchen/go-httpbin
kubectl -n httpbin expose deployment httpbin --name httpbin-svc --port 8080

kubectl create namespace nginx
kubectl label namespace nginx shared-gateway-access=true
kubectl -n nginx create deployment nginx --image=nginx:alpine
kubectl -n nginx expose deployment nginx --name nginx-svc --port 80

HTTPRoute

The HTTPRoute defines how you want to match a request based on conditions (matches), process it (filters), and forward the request to a backend service (backendRefs). With everything set up, let’s delve into the HTTPRoute API specification to explore its capabilities.

HTTPRouteMatch

Let’s explore the criteria that can be utilized for matching requests.

HTTPPathMatch

Similar to using Ingress, we can use annotations on HTTPRoute to enhance routing capability.

The first criteria is to match requests path. As we can see from below examples, path supports Exact, PathPrefix and RegularExpression. This is similar to the Ingress Path types except for the Regex one.

• Exact

  apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    name: nginx-route
    namespace: nginx
    annotations:
      konghq.com/strip-path: 'true'
  spec:
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: kong-gw-demo
        namespace: kong
    rules:
      - matches:
        - path:
            type: Exact
            value: /nginx
        backendRefs:
        - name: nginx-svc
          kind: Service
          port: 80

• PathPrefix

  apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    name: httpbin-route
    namespace: httpbin
    annotations:
      konghq.com/strip-path: 'true'
  spec:
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: kong-gw-demo
        namespace: kong
    rules:
      - matches:
        - path:
            type: PathPrefix
            value: /test
        backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

• RegularExpression

  apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    name: httpbin-regex-route
    namespace: httpbin
    annotations:
      konghq.com/strip-path: 'true'
  spec:
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: kong-gw-demo
        namespace: kong
    rules:
      - matches:
        - path:
            type: RegularExpression
            value: /api/v(1|2)
        backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

HTTPHeaderMatch

We can match request by header on HTTPRoute directly now. You need to use konghq.com/headers.* annotation to achieve the same on Ingress. Header matching support Exact and RegularExpression.

• Exact

  apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    name: httpbin-route
    namespace: httpbin
    annotations:
      konghq.com/strip-path: 'true'
  spec:
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: kong-gw-demo
        namespace: kong
    rules:
      - matches:
        - path:
            type: PathPrefix
            value: /test
          headers:
          - name: "x-version"
            type: Exact
            value: "2"
        backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

• RegularExpression

  apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    name: httpbin-route
    namespace: httpbin
    annotations:
      konghq.com/strip-path: 'true'
  spec:
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: kong-gw-demo
        namespace: kong
    rules:
      - matches:
        - path:
            type: PathPrefix
            value: /test
          headers:
          - name: "x-version"
            type: RegularExpression
            value: "(1|2)"
        backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

HTTPQueryParamMatch

This is supported by using expressions router flavour only.

With expressions and KIC, we can also match request by query string. Same as match by header, query string match supports Exact and RegularExpression.

• Exact

  apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    name: httpbin-route
    namespace: httpbin
    annotations:
      konghq.com/strip-path: 'true'
  spec:
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: kong-gw-demo
        namespace: kong
    rules:
      - matches:
        - path:
            type: PathPrefix
            value: /test
          queryParams:
          - type: Exact
            name: version
            value: "2"
        backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

• RegularExpression

  apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    name: httpbin-route
    namespace: httpbin
    annotations:
      konghq.com/strip-path: 'true'
  spec:
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: kong-gw-demo
        namespace: kong
    rules:
      - matches:
        - path:
            type: PathPrefix
            value: /test
          queryParams:
          - type: RegularExpression
            name: version
            value: (1|2)
        backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

HTTPMethod

Continuing, we can also match requests based on the HTTP method. In the following example, we match GET and POST methods at the path /test.

Note: There is a limitation to match one method per match path. In my opinion it should support multiple methods (as an array) but this is considered a breaking change hence it is not being done at the moment. GitHub issue

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin-route
  namespace: httpbin
  annotations:
    konghq.com/strip-path: 'true'
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: kong-gw-demo
      namespace: kong
  rules:
    - matches:
      - path:
          type: PathPrefix
          value: /test
        method: "POST"
      - path:
          type: PathPrefix
          value: /test
        method: "GET"
      backendRefs:
      - name: httpbin-svc
        kind: Service
        port: 8080

Host name Match

If we want to use host name to the matching rule, we can put the host names on the HTTPRoute object. As we did not specify hostname on our Gateway Listener, the listener allows all hosts names to be matched. In the example below, the HTTPRoute can match all three domain names proxy.li.k8s, test.proxy.li.k8s or test.proxy.li.test with path /test.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin-route
  namespace: httpbin
  annotations:
    konghq.com/strip-path: 'true'
spec:
  hostnames:
  - proxy.li.k8s
  - test.proxy.li.k8s
  - test.proxy.li.test
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: kong-gw-demo
      namespace: kong
  rules:
    - matches:
      - path:
          type: PathPrefix
          value: /test
      backendRefs:
      - name: httpbin-svc
        kind: Service
        port: 8080

The cluster operator can also impose restrictions on the hostnames that the listener should respond to. By adding hostnames to the listener section, only the allowed hostnames will be matched to the HTTPRoute. Hostnames prefixed with a wildcard label (*) are interpreted as a suffix match. This means that a match for *.example.com would include both test.example.com and foo.test.example.com, but not example.com.

Now, let’s update our Gateway object kong-gw-demo to include hostname: "*.li.k8s", above HTTPRoute will only match the requests that has path /test and hostname proxy.li.k8s or test.proxy.li.k8s, not test.proxy.li.test.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kong-gw-demo
  namespace: kong
spec:
  gatewayClassName: kong-demo
  listeners:
    - name: proxy
      hostname: "*.li.k8s"
      port: 80
      protocol: HTTP
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              shared-gateway-access: "true"

Here is a table explaining how hostname match on Gateway and HTTPRoute.

Gateway	HTTPRoute	Matched Hostname
proxy.li.k8s	*.li.k8s	proxy.li.k8s
————–	—————–	——————
*.li.k8s	*.li.k8s	*.li.k8s
————–	—————–	——————
*.li.k8s	proxy.li.k8s	proxy.li.k8s
	test.proxy.li.k8s	test.proxy.li.k8s
	proxy.li.net	Not match

HTTPRouteFilter

HTTPRouteFilter helps to process request or respoonse right through the request lifecycle. While Kong plugins offer additional capabilities, here we’ll delve into what can be accomplished using the native Gateway API.

RequestHeaderModifier

As name suggested, you can modify request headers directly on the HTTPRoute. It supports three actions:

• set: Set overwrites the request with the given header (name, value) before the action.
• add: Add adds the given header(s) (name, value) to the request before the action. It appends to any existing values associated with the header name.
• remove: Remove the given header(s) from the HTTP request before the action. The value of Remove is a list of HTTP header names. Note that the header names are case-insensitive.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin-route
  namespace: httpbin
  annotations:
    konghq.com/strip-path: "true"
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: kong-gw-demo
      namespace: kong
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /test
      filters:
        - type: RequestHeaderModifier
          requestHeaderModifier:
            add:
              - name: x-test-id
                value: fomm
            set: 
              - name: User-Agent
                value: fomm-local
            remove: ["Accept"]
      backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

ResponseHeaderModifier

Similar to RequestHeaderModifier, you can also change Response header. It also support the same 3 actions, set,add and remove.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin-route
  namespace: httpbin
  annotations:
    konghq.com/strip-path: "true"
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: kong-gw-demo
      namespace: kong
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /test
      filters:
        - type: ResponseHeaderModifier
          responseHeaderModifier:
            add:
              - name: x-test-id
                value: fomm
            set: 
              - name: User-Agent
                value: fomm-local
            remove: ["Access-Control-Allow-Origin", "Access-Control-Allow-Credentials"]
      backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

RequestRedirect

This filter is designed to short-circuit the request and redirect it to an entirely different destination. Consequently, the backendRef cannot be used with this filter, as it doesn’t align with the concept of redirection. If you don’t specify the path, port, host, etc., it will utilize the original request information. In the example below, the request is short-circuited to httpbin server.

There are two types, ReplacePrefixMatch and ReplaceFullPath. However, from testing it seems Kong Ingress Controller does not support ReplacePrefixMatch at the moment. You can find ReplaceFullPath below.

• ReplaceFullPath

  apiVersion: gateway.networking.k8s.io/v1
  kind: HTTPRoute
  metadata:
    name: httpbin-route
    namespace: httpbin
    annotations:
      konghq.com/strip-path: "true"
  spec:
    parentRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: kong-gw-demo
        namespace: kong
    rules:
      - matches:
        - path:
            type: PathPrefix
            value: /test
        filters:
        - type: RequestRedirect
          requestRedirect:
            scheme: http
            hostname: httpbin.org
            path:
              type: ReplaceFullPath
              replaceFullPath: /anything/paprika
            statusCode: 301
            port: 80

ExtensionRef

ExtensionRef is an implementation-specific extension to the filter behavior. For Kong Ingress Controller we can use it to apply KongPlugin on the HTTPRoute. This is similar to adding konghq.com/plugins annotation.

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: key-auth-example
  namespace: httpbin
plugin: key-auth
config:
  key_names:
  - apikey
---
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: acl-example
  namespace: httpbin
plugin: acl
config:
  allow:
  - group1
  hide_groups_header: true
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin-route
  namespace: httpbin
  annotations:
    konghq.com/strip-path: "true"
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: kong-gw-demo
      namespace: kong
  rules:
    - matches:
      - path:
          type: PathPrefix
          value: /test
      filters:
        - type: ExtensionRef
          extensionRef:
            group: configuration.konghq.com
            kind: KongPlugin
            name: key-auth-example
        - type: ExtensionRef
          extensionRef:
            group: configuration.konghq.com
            kind: KongPlugin
            name: acl-example
      backendRefs:
        - name: httpbin-svc
          kind: Service
          port: 8080

BackendRefs

BackendRef is a reference to a backend where the request should be forwarded to.

Cross-namespace routing

As mentioned earlier, the Ingress object cannot route requests to a service in a different namespace. However, Gateway API has changed this limitation, allowing users to route traffic to services in different namespaces. To achieve this, we need to use ReferenceGrant to permit requests to flow through.

I need to patch namespace kong to have label shared-gateway-access=true so KIC will response to the HTTPRoute in namespace kong.

kubectl label namespace kong shared-gateway-access=true

Next we can create our HTTPRoute in kong namespace and route request to httpbin service in httpbin namespace.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin-route
  namespace: kong
  annotations:
    konghq.com/strip-path: "true"
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: kong-gw-demo
      namespace: kong
  hostnames:
    - proxy.li.k8s
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /test
      backendRefs:
        - name: httpbin-svc
          namespace: httpbin
          kind: Service
          port: 8080
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: httpbin-grant-kong
  namespace: httpbin
spec:
  from:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    namespace: kong
  to:
  - group: ""
    kind: Service

Weighted routing

Another useful built-in feature is traffic splitting by setting weights on different backends. In the example below, traffic is split 90% to httpbin and 10% to nginx.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: nginx-httpbin-route
  namespace: kong
  annotations:
    konghq.com/strip-path: 'true'
spec:
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: kong-gw-demo
      namespace: kong
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /test
    backendRefs:
    - name: httpbin-svc
      namespace: httpbin
      kind: Service
      namespace: httpbin
      port: 8080
      weight: 90
    - name: nginx-svc
      kind: Service
      namespace: nginx
      port: 80
      weight: 10
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: httpbin-grant-kong
  namespace: httpbin
spec:
  from:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    namespace: kong
  to:
  - group: ""
    kind: Service
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
  name: nginx-grant-kong
  namespace: nginx
spec:
  from:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    namespace: kong
  to:
  - group: ""
    kind: Service

Summary

Kubernetes Gateway API is undoubtedly the future, and I encourage you to start exploring this new standard. Many more features are continuously being added too.

There’s still a lot for us to delve into, including TLS termination and cert-manager integration, which significantly differ from Ingress. Additionally, L4 routing is another aspect we may want to discuss, but it’s probably better to cover these topics in separate posts.

That’s all I want to share with you today, and I look forward to seeing you next time.