Skip to content

Kubernetes Gateway in Kubernetes and Control Plane in a VM

Overview

In this approach, APK Dataplane is deployed in a kubernetes cluster and APIM in Virtual Machine. In this pattern APK dataplane will have visibility over all the namespace (or a predefined set of namespaces) in the cluster.

ap_cp_to_dp_vms

Advantages

  • Isolation: APK Dataplane operates in its own dedicated kubernetes cluster, while APIM CP in a VM, providing the highest level of isolation and minimizing the risk of interference between components.
  • Security: With separate cluster and VM setup it's easier to implement stringent security policies specific to each component.
  • Performance Predictability: Resources are exclusively allocated to each cluster, guaranteeing consistent performance levels.

Considerations

  • Resource Utilization: This approach may require additional resources due to the need for separate cluster and VM for each environment.
  • Administrative Overhead: Managing multiple cluster and VMs can be more complex and may require additional administrative effort.

Installation guideline

By default, WSO2 APK is configured to support the Data Plane Only Pattern. You can follow the below steps to do the deployment using this pattern.

Create Kubernetes namespace

In this guide, the APIM Control Plane and the Kubernetes Gateway are installed in the same namespace. If you prefer, you can use any standalone Kubernetes Gateway pattern to deploy the Kubernetes Gateway.

kubectl create ns apk

Set up WSO2 Kubernetes Gateway 1.3.0

  1. Create a new Helm repository with the latest Kubernetes Gateway release using the following command. Let's consider the <repository-name> as wso2apk.

    console helm repo add wso2apk https://github.com/wso2/apk/releases/download/1.3.0-1

  2. Execute the following command to update the helm repositories.

    console helm repo update

  3. Download the values.yaml file

    To obtain the values.yaml file, use the helm show values command. Replace <repository-name> with the actual repository name and <version-of-APK> with the desired version of the Kubernetes Gateway. Run the following command:

    helm show values wso2apk/apk-helm --version 1.3.0-1 > kg-values.yaml

    helm show values <repository-name>/apk-helm --version <version-of-APK> > kg-values.yaml

  4. To add Kubernetes Gateway as a gateway, add the following configurations to kg-values.yaml.

    • Add the following configuration under the wso2.apk section

      yaml cp: enabledSubscription: true host: "apim-apk-agent-service.apk.svc.cluster.local" skipSSLVerification: true

      enabledSubscription This field must be set to true to retrieve subscription details at the gateway level.
      host Agent hostname.

      {agentService}.{namespace}.svc.cluster.local

      You can retrieve the agentService name using kubectl get svc -n <namespace>

      At this point, the agent is not configured, so you can keep the default value as is. Once configured, ensure you update this value and perform a Helm upgrade.

      skipSSLVerification Skip SSL verification between the Agent and Gateway.

    • (Optional) Change the default listener hostnames

      By default, the Kubernetes Gateway has 2 listeners:

      • System APIs Listener - api.am.wso2.com
      • Gateway Listener - gw.wso2.com

      If you wish to change the default hostname and vhost, modify the following values.yaml configurations. For example, if you want to deploy a production environment with the domain name example.com, expose your APIs through prod.gw.example.com, and expose APK system APIs through prod.apk.example.com, configure as follows:

      wso2.apk.listener.hostname: 'prod.apk.example.com'
wso2.apk.dp.gateway.listener.hostname: 'gw.example.com'
wso2.apk.dp.configdeployer.vhosts: [{"hosts":["gw.example.com"],"name":"prod","type":"production"}]

    • Configure API Key Issuer

      If you want to use API Key Authentication, you can enable it by changing the following configuration in kg-values.yaml under wso2.apk.dp.gatewayRuntime.deployment.enforcer.configs section.

      yaml apiKey: enabled: true issuer: "https://am.wso2.com:443/oauth2/token"

      enabled Enable API Key Authentication
      issuer API Key Issuer URL. This is used to issue API Keys for the APIs.

  5. Install the Helm chart

    To begin the installation, run the following command.

    helm install apk wso2apk/apk-helm --version 1.3.0 -f kg-values.yaml -n apk

    helm install <chart-name> <repository-name>/apk-helm --version <version-of-APK> -f <path-to-values.yaml-file> -n <namespace>

Set up WSO2 API Manager Control Plane 4.5.0

Set up WSO2 API Manager 4.5.0 in a VM.

Refer to the APIM Deployment Guide to deploy the APIM 4.5.0 in a VM. Change the configurations in the deployment.toml file as below.

  1. Configure supported gateway types in API Manager.

    Add the following configuration to the deployment.toml file to provide supported gateway types to API Manager. Here, need to provide the supported gateway types as APK.

    toml [apim] gateway_type = "Regular,APK"

    toml [apim] gateway_type = "APK"

  2. Add the following configuration to the deployment.toml file to provide Kubernetes gateway to API Manager.

    toml [[apim.gateway.environment]] name = "Default_APK" type = "hybrid" gateway_type = "APK" provider = "wso2" display_in_api_console = true description = "This is a hybrid gateway that handles both production and sandbox token traffic." show_as_token_endpoint_url = true http_endpoint = "http://default.gw.wso2.com:9090" https_endpoint = "https://default.gw.wso2.com:9095"

    name Gateway Name. This name is required during Kubernetes gateway agent configuration
    type To handle both production and sandbox token traffic, set this to hybrid.
    gateway_type To identify the gateway as the Kubernetes Gateway, this type is required to be set as APK.
    provider Gateway provider.
    https_endpoint

    This setting is critical for the Control Plane to locate the Gateway listener. If you change the Gateway listener in the Kubernetes Gateway configuration, that change should be reflected here. The Gateway listener hostname must be configured as default.{gateway.listener.hostname}:9095, where default and 9095 are fixed values.

Set up WSO2 Kubernetes Gateway Agent 1.3.0

Note

The Kubernetes Gateway Agent provides a connection between the APIM Control Plane and the Kubernetes Gateway.

  1. Create a new Helm repository with the latest Kubernetes Gateway Agent release using the following command. Let's consider the <repository-name> as wso2apkagent for this guide.

    console helm repo add wso2apkagent https://github.com/wso2/product-apim-tooling/releases/download/1.3.0

  2. Execute the following command to update the helm repositories.

    console helm repo update

  3. Download values.yaml File

    To obtain the values.yaml file, you can use the helm show values command. Replace <repository-name> with the actual repository name and <version-of-Agent> with the desired version of the Kubernetes Gateway Agent. Run the following command:

    helm show values wso2apkagent/apim-apk-agent --version 1.3.0 > kg-agent-values.yaml

    helm show values <repository-name>/apk-helm --version <version-of-APK> > kg-agent-values.yaml

  4. Configuration required to connect the Kubernetes Gateway Agent in kg-agent-values.yaml

    • Configure Control Plane (APIM) related configuration in the Kubernetes Gateway Agent

      yaml controlPlane: enabled: true serviceURL: https://apim-wso2am-cp-1-service.apk.svc.cluster.local:9443/ username: admin password: admin environmentLabels: Default skipSSLVerification: true eventListeningEndpoints: amqp://admin:[email protected]:5672?retries='10'&connectdelay='30' internalKeyIssuer: https://am.wso2.com:443/oauth2/token

      serviceURL Service URL of the API Manager Control Plane.

      Format: {apimService}.{namespace}.svc.cluster.local

      You can retrieve the apimService name using kubectl get svc -n <namespace>

      environmentLabels Gateway environment label is the environment name that you define under the environments section in the API Manager values.yaml file, which sets up the Kubernetes Gateway. In this scenario, it's Default_APK.
      eventListeningEndpoints Event listening endpoint in the API Manager.
      Username and Password Admin credentials of the API Manager.
      internalKeyIssuer Internal Key Issuer URL of the API Manager. This is used to issue internal keys for the APIs.

    • Configure data plane (APK Gateway) related configuration in the Kubernetes Gateway Agent

      yaml dataPlane: enabled: true k8ResourceEndpoint: https://apk-wso2-apk-config-ds-service.apk.svc.cluster.local:9443/api/configurator/apis/generate-k8s-resources namespace: apk

      k8ResourceEndpoint Config deployer service endpoint.

      Format: {configDeployerService}.{namespace}.svc.cluster.local:{port}/api/configurator/apis/

      You can retrieve the configDeployerService name using kubectl get svc -n <namespace>

      namespace Namespace where the Kubernetes Gateway is deployed.

    • Change the mode of the agent configuration as shown below.

      yaml agent: mode: CPtoDP

  5. Install the Kubernetes Gateway Agent

    Install the Kubernetes Gateway Agent components and start WSO2 API Platform For Kubernetes. Consider apk as the <chart-name> for this guide. For the --version parameter, use the version corresponding to the release from step 1. The deployment will take a few minutes to complete.

    helm install apim-apk-agent wso2apkagent/apim-apk-agent --version 1.3.0 -f kg-agent-values.yaml -n apk

    helm install <chart-name> <repository-name>/apim-apk-agent --version <version-of-APK-Agent> -f <path-to-values.yaml-file> -n namespace

Verify the Deployment

You can verify the deployment by executing the following command. Once completed, you will see the status of the pods as shown below.

kubectl get pods -n apk

Pod Status

Configure the Hosts File

kubectl get svc -n apk Pod Status

kubectl get ing -n apk

Pod Status

Add hostname mappings to the /etc/hosts file as follows:

Domain name IP
am.wso2.com (APIM Control Plane ingress hostname) 135.171.24.117 (As per above image). You can find it using kubectl get ing -n <namespace>
default.gw.wso2.com (This should be the same as the value you added for the Gateway Listener) 135.171.24.26 (As per above image). You can find it using kubectl get svc -n <namespace>
api.am.wso2.com (This should be the same as the value you added for the System APIs Listener) 135.171.24.26 (As per above image). You can find it using kubectl get svc -n <namespace>
idp.am.wso2.com (You don't need to change this) 135.171.24.26 (As per above image). You can find it using kubectl get svc -n <namespace>

This approach is recommended for all Production, Dev, Test, and UAT environments. Here a seperate kubernetes cluster and a VM server is required for each environment.