Skip to main content
Version: v1.2 🚧

RuntimeProxy

Summary​

KoordRuntimeProxy acts as a proxy between kubelet and containerd(dockerd under dockershim scenario), which is designed to intercept CRI request, and apply some resource management policies, such as setting different cgroup parameters by pod priorities under hybrid workload orchestration scenario, applying new isolation policies for latest Linux kernel, CPU architecture, and etc.

There are two components involved, KoordRuntimeProxy and RuntimePlugins.

image

Goals​

  • Enhance resource management for QoS based Scheduling.
  • Provide interface for new isolation features which are not supported by CRI.

Components​

KoordRuntimeProxy​

KoordRuntimeProxy is in charge of intercepting request during pod's lifecycle, such as RunPodSandbox, CreateContainer etc., and then calling RuntimePlugins to do resource isolation policies before transferring request to backend containerd(dockerd) and after transferring response to kubelet. KoordRuntimeProxy provides an isolation-policy-execution framework which allows customized plugins registered to do specified isolation policies, these plugins are called RuntimePlugins. KoordRuntimeProxy itself does NOT do any isolation policies.

RuntimePlugins​

RuntimePlugins register events(RunPodSandbox etc.) to KoordRuntimeProxy and would receive notifications when events happen. RuntimePlugins should complete resource isolation policies basing on the notification message, and then response KoordRuntimeProxy, KoordRuntimeProxy would decide to transfer request to backend containerd or discard request according to plugins' response.

If no RuntimePlugins registered, KoordRuntimeProxy would become a transparent proxy between kubelet and containerd.

Architecture​

image

There are 4 main components for KoordRuntimeProxy.

CRI Server​

As a proxy between kubelet and containerd, KoordRuntimeProxy acts as a CRI server for kubelet(http server under dockershim scenario). It should intercept all request from kubelet, and generate protocols for talking with plugins before and after talking with backend containerd(dockerd)

Plugins Manager​

PluginsManager is in charge of parsing registered plugin info from /etc/runtime/hookserver.d dynamically.

Runtime Dispatcher​

RuntimeDispatcher is designed to manage communications with plugins.

Store​

As a proxy, KoordRuntimeProxy had better be designed as stateless, but sometimes it does NOT work. Take StartContainer hook for example, there exists only containerID in CRI StartContainerRequest, which is not enough for plugins to adapt policies since plugins may not store pod/container info(such as meta, priority) locally. So KoordRuntimeProxy should store pod/container info during RunPodSandbox/CreateContainer Stage. When StartContainer request comes, KoordRuntimeProxy can get pod/container info by containerID, and then call plugins with pod/container info.

With store, there would be pod/container info everytime KoordRuntimeProxy calls plugins, so there is no need for plugins to store pod/container info exceptionally, plugins can be designed as stateless.

Considering performance, store locates in memory and does not generate external io to disk.

Runtime Plugins​

How to Register Plugins​

All the plugin config files should be put to /etc/runtime/hookserver.d with .json suffix. You can register the plugin implemented by koordlet with RuntimeProxy:

  1. touch /etc/runtime/hookserver.d/koordlet.json
  2. Copy the following content into /etc/runtime/hookserver.d/koordlet.json
{
"remote-endpoint": "/var/run/koordlet/koordlet.sock",
"failure-policy": "Ignore",
"runtime-hooks": [
"PreRunPodSandbox",
"PreCreateContainer",
"PreStartContainer"
]
}

There are 3 fields involved:

  • remote-endpoint: endpoint KoordRuntimeProxy talking with plugin, generated by plugin.
  • failure-policy: policy when calling plugin fail, Fail or Ignore, default to Ignore.
  • runtime-hooks: currently 7 hook points:
    1. PreRunPodSandbox
    2. PreCreateContainer
    3. PreStartContainer
    4. PostStartContainer
    5. PreUpdateContainerResources
    6. PostStopContainer
    7. PostStopPodSandbox

hook points with prefix 'Pre' means calling plugins before transferring request to contianerd(dockerd). hook points with prefix 'Post' means calling plugins after receiving response from containerd(dockerd). plugin provider can set any hook combinations to "runtime-hooks".

Protocols between KoordRuntimeProxy and Plugins​

Protocols

Examples for Runtime Plugins​

koordlet-runtime-plugin-design

Installation​

Installing from sources​

get sources: git clone https://github.com/koordinator-sh/koordinator.git

build: cd koordinator; make build-koord-runtime-proxy

Installing from packages​

Download latest released package from: https://github.com/koordinator-sh/koordinator/releases

Setup Kubelet​

Under containerd scenario, to make koord-runtime-proxy a proxy between kubelet and containerd, kubelet parameters should be altered as shown below:

kubelet <other options> --container-runtime=remote --container-runtime-endpoint=unix:///var/run/koord-runtimeproxy/runtimeproxy.sock

Under docker scenario, to make koord-runtime-proxy a proxy between kubelet and dockerd, kubelet parameters should be altered as shown below:

kubelet <other options> --docker-endpoint=unix:///var/run/koord-runtimeproxy/runtimeproxy.sock

Setup KoordRuntimeProxy​

Firstly, please make sure your runtime backend is containerd or dockerd.

Under containerd scenario, koord-runtime-proxy can be setup with command:

koord-runtime-proxy --remote-runtime-service-endpoint=<runtime sockfile path>
--remote-image-service-endpoint=<image sockfile path>

If containerd listening CRI request on default /var/run/koord-runtimeproxy/runtimeproxy.sock, koord-runtime-proxy can be setup by:

koord-runtime-proxy

Under docker scenario, koord-runtime-proxy should be setup with the additional parameter --backend-runtime-mode Docker, and without remote-image-service-endpoint:

koord-runtime-proxy --backend-runtime-mode=Docker --remote-runtime-service-endpoint=<runtime sockfile path>