How to Use Request Transformation Advanced Plugin

, , ,

Kong is so powerful and flexible because of its plugins. When users are designing their APIs, sometimes transforming the request is inevitable. Request Transformer Advanced plugin comes in very handy in such situation. In this article I will try my best to explain what you can do with this plugin. As it is a complicated plugin, I will only cover a few examples, users need to decide what they want to do with it.

What can be transform?

Please see below table of what can be transformed. Let’s say if you want to add a header, when you are enabling this plugin on a route, service or globally, you need to add the new header to config.add.headers parameter.

headers body querystring uri http_method
add add add
remove remove remove
replace replace replace replace
rename rename rename
append append append
allow

Preparation

Let me start with some examples. I will be using Admin API to set up.

Prerequisite :

A Kong instance with database.

You can find official documentation here.

Create a service

I am using ealen/echo-server as my upstream service.

1
2
3
4
curl -X POST 'http://localhost:8001/services' \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, */*" \
  -d '{"name":"request-service","url":"http://echo"}'

Create Route

1
2
3
4
curl -X POST 'http://localhost:8001/services/request-service/routes' \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, */*" \
  -d '{"name":"request-route","paths":["/request"]}'

Now when we visit our route at curl 'http://localhost:8000/request' -i, we should get HTTP/1.1 200 OK.

Usage Example

I will add plugin globally for consistent results. Users can enable it on a server or a route.

Basic Usage

Next I will use this plugin to

  • Add a new header - test:addedHeader
  • Remove body parameter - ‘testBody’
  • Replace Uri - from /request to /new-request
1
2
3
4
5
curl -X POST 'http://localhost:8001/plugins' \
  -d "name=request-transformer-advanced" \
  -d "config.add.headers=test:addedHeader" \
  -d "config.remove.body=testBody" \
  -d "config.replace.uri=/new-request"

Let’s make a call to our routes, which I POST a json input to echo server.

1
2
3
curl -X POST 'http://localhost:8000/request' \
  -H "Content-Type: application/json" \
  -d '{"testBody":"missing","newBody":"keep"}'

We will see this plugin did exactly what we want it to do.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
{
  "http": {
    "method": "POST",
    "baseUrl": "",
    "originalUrl": "/new-request",
    "protocol": "http"
  },
  "request": {
    "params": {
      "0": "/new-request"
    },
    "query": {},
    "cookies": {},
    "body": {
      "newBody": "keep"
    },
    "headers": {
      "host": "echo",
      "test": "addedHeader",
      "connection": "keep-alive",
      "x-forwarded-for": "172.28.0.1",
      "x-forwarded-proto": "http",
      "x-forwarded-host": "localhost",
      "x-forwarded-port": "8000",
      "x-forwarded-path": "/request",
      "x-forwarded-prefix": "/request",
      "x-real-ip": "172.28.0.1",
      "content-length": "18",
      "user-agent": "curl/7.64.1",
      "accept": "*/*",
      "content-type": "application/json"
    }
  }
}

Advanced Usage

For the advanced usage, I will cover how to use build in variables as well as Lua code to transform requests.

Template as Value

There are 3 sandboxed variables that we can use to make this plugin more useful.

  • headers
  • query_params
  • uri_captures

Official documentation gave us some use case, let me use one of them as an example.

1
2
3
curl -X POST 'http://localhost:8001/plugins' \
  -d 'name=request-transformer-advanced' \
  -d 'config.add.headers=Class:$(query_params["category"] or headers["category"] or "standard")'

In this example we will add a Class header to the request. The value of this header comes from another header or query string category. If there is no such header or querystring, we use standard to Class header.

Let’s try and see if it works.

1
2
curl 'http://localhost:8000/request' \
  -H "category: silver"

We should get below result. we can see "class": "silver" is added to the upstream header.

We can also pass querystring as curl 'http://localhost:8000/request?category=silver' and we should see the same header added.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
{
  "http": {
    "method": "GET",
    "baseUrl": "",
    "originalUrl": "/",
    "protocol": "http"
  },
  "request": {
    "params": {
      "0": "/"
    },
    "query": {},
    "cookies": {},
    "body": {},
    "headers": {
      "host": "echo",
      "connection": "keep-alive",
      "x-forwarded-for": "192.168.186.1",
      "x-forwarded-proto": "http",
      "x-forwarded-host": "localhost",
      "x-forwarded-port": "8000",
      "x-forwarded-path": "/request",
      "x-forwarded-prefix": "/request",
      "x-real-ip": "192.168.186.1",
      "user-agent": "curl/7.78.0",
      "accept": "*/*",
      "category": "silver",
      "class": "silver"
    }
  }
}

Advanced templates function

Please note this feature is turned off by default if you are using Kong 2.2.x or later. Please check changelog for more information.
I have set KONG_UNTRUSTED_LUA=on for the demonstration purpose.

If you want to go further and use logical operations to transform the requests, you can write some simple Lua code as advanced templates. Here is one use case.

Let’s say you want to add a header called Country based on the query string brand passed in.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
curl -X POST 'http://localhost:8001/plugins' \
-d 'name=request-transformer-advanced' \
-d 'config.add.headers=Country:$((function()
    local value = query_params.brand
    if not value then
      return "Unknown"
    end
    value = value:lower()
    if value == "telstra" then
      return "Australia"
    elseif value == "bmw" then
        return "Germany"
    else
      return "Unknown"
    end
  end)())'

Now when I run curl 'http://localhost:8000/request?brand=telstra', you can see the Country header is added with Australia as its value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
{
  "http": {
    "method": "GET",
    "baseUrl": "",
    "originalUrl": "/?brand=telstra",
    "protocol": "http"
  },
  "request": {
    "params": {
      "0": "/"
    },
    "query": {
      "brand": "telstra"
    },
    "cookies": {},
    "body": {},
    "headers": {
      "host": "echo",
      "connection": "keep-alive",
      "x-forwarded-for": "192.168.186.1",
      "x-forwarded-proto": "http",
      "x-forwarded-host": "localhost",
      "x-forwarded-port": "8000",
      "x-forwarded-path": "/request",
      "x-forwarded-prefix": "/request",
      "x-real-ip": "192.168.186.1",
      "user-agent": "curl/7.78.0",
      "accept": "*/*",
      "country": "Australia"
    }
  }
}

Use uri_captures variables

Kong supports regular expression pattern matching for an Route’s paths field. These Regex match group can be captured and used in Request Transformation plugin as well.

Here is an example.

1
2
3
4
curl -X POST 'http://localhost:8001/services/request-service/routes' \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, */*" \
  -d '{"name":"regex-route","paths":["/myapi/(?<apiVersion>v[0-3])/user/(?<userID>[0-9]{6})"]}'

When user consume this API, the apiVersion and userID is captured and can be used with Request Transformer Advanced Plugin.

Let’s say I want to add apiVersion and userID to my request header, we can enable plugins as below.

1
2
3
4
curl -X POST 'http://localhost:8001/plugins' \
  -d 'name=request-transformer-advanced' \
  -d 'config.add.headers=USER:$(uri_captures["userID"])' \
  -d 'config.add.headers=API:$(uri_captures["apiVersion"])'

Now when we access the route at curl 'http://localhost:8000/myapi/v2/user/123456/test', we should get below response. As we can see "api": "v2" and "user": "123456" were sent to upstream.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
{
  "http": {
    "method": "GET",
    "baseUrl": "",
    "originalUrl": "/test",
    "protocol": "http"
  },
  "request": {
    "params": {
      "0": "/test"
    },
    "query": {},
    "cookies": {},
    "body": {},
    "headers": {
      "host": "echo",
      "connection": "keep-alive",
      "x-forwarded-for": "192.168.186.1",
      "x-forwarded-proto": "http",
      "x-forwarded-host": "localhost",
      "x-forwarded-port": "8000",
      "x-forwarded-path": "/myapi/v2/user/123456/test",
      "x-forwarded-prefix": "/myapi/v2/user/123456",
      "x-real-ip": "192.168.186.1",
      "user-agent": "curl/7.78.0",
      "accept": "*/*",
      "api": "v2",
      "user": "123456"
    }
  }
}

Other deployments methods

This plugin is compatible for Kong deployed in DBless mode.

DBless deployment

Please save below content to kong.yaml and start Kong with it.

I will use the example from Advanced templates function section here because it is the most complicated one.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
_format_version: "2.1"
_transform: true
services:
- name: request-service
  url: https://httpbin.org/anything
  routes:
  - name: request-route
    paths:
    - /request
plugins:
- name: request-transformer-advanced
  route: request-route
  config:
    add:
      headers:
      - |-
        Country:$((function()
            local value = query_params.brand
            if not value then
              return "Unknown"
            end
            value = value:lower()
            if value == "telstra" then
              return "Australia"
            elseif value == "bmw" then
              return "Germany"
            else
              return "Unknown"
            end
          end)())

Kubernetes Ingress Controller

You can also applied this plugin with Kong Ingress Controller.

Below example will deploy:

  • Echo deployment
  • Echo Service
  • Request Transformer Advanced plugin with advanced template
  • Ingress rule to use this plugin

Please save below to request.yaml and then kubectl apply -f request.yaml to apply it.

Now you should get the same result as above.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo-pod
  template:
    metadata:
      labels:
        app: echo-pod
    spec:
      containers:
      - name: echoheaders
        image: k8s.gcr.io/echoserver:1.10
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: echo-service
spec:
  selector:
    app: echo-pod
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 8080
  type: LoadBalancer
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: proxy-to-echo
  annotations:
    konghq.com/plugins: test-transform
    konghq.com/strip-path: "true"
spec:
  ingressClassName: kong
  rules:
  - http:
      paths:
      - backend:
          service:
            name: echo-service
            port:
              number: 80
        path: /request
        pathType: Prefix
---
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: test-transform
plugin: request-transformer-advanced
config:
  add:
    headers:
    - 'Country:$((function()
      local value = query_params.brand
      if not value then
        return "Unknown"
      end
      value = value:lower()
      if value == "telstra" then
        return "Australia"
      elseif value == "bmw" then
          return "Germany"
      else
        return "Unknown"
      end
    end)())'

This is all I want to talk about today, thanks for reading, see you next time.