RuntimeProxy

Summary

KoordRuntimeProxy acts as a proxy between kubelet and containerd(dockerd under dockershim scenario), which is designed to intercept CRI request, and apply some resource management policies, such as setting different cgroup parameters by pod priorities under hybrid workload orchestration scenario, applying new isolation policies for latest Linux kernel, CPU architecture, and etc.

There are two components involved, KoordRuntimeProxy and RuntimePlugins.

Goals

• Enhance resource management for QoS based Scheduling.
• Provide interface for new isolation features which are not supported by CRI.

Components

KoordRuntimeProxy

KoordRuntimeProxy is in charge of intercepting request during pod's lifecycle, such as RunPodSandbox, CreateContainer etc., and then calling RuntimePlugins to do resource isolation policies before transferring request to backend containerd(dockerd) and after transferring response to kubelet. KoordRuntimeProxy provides an isolation-policy-execution framework which allows customized plugins registered to do specified isolation policies, these plugins are called RuntimePlugins. KoordRuntimeProxy itself does NOT do any isolation policies.

RuntimePlugins

RuntimePlugins register events(RunPodSandbox etc.) to KoordRuntimeProxy and would receive notifications when events happen. RuntimePlugins should complete resource isolation policies basing on the notification message, and then response KoordRuntimeProxy, KoordRuntimeProxy would decide to transfer request to backend containerd or discard request according to plugins' response.

If no RuntimePlugins registered, KoordRuntimeProxy would become a transparent proxy between kubelet and containerd.

Architecture

There are 4 main components for KoordRuntimeProxy.

CRI Server

As a proxy between kubelet and containerd, KoordRuntimeProxy acts as a CRI server for kubelet(http server under dockershim scenario). It should intercept all request from kubelet, and generate protocols for talking with plugins before and after talking with backend containerd(dockerd)

Plugins Manager

PluginsManager is in charge of parsing registered plugin info from /etc/runtime/hookserver.d dynamically.

Runtime Dispatcher

RuntimeDispatcher is designed to manage communications with plugins.

Store

As a proxy, KoordRuntimeProxy had better be designed as stateless, but sometimes it does NOT work. Take StartContainer hook for example, there exists only containerID in CRI StartContainerRequest, which is not enough for plugins to adapt policies since plugins may not store pod/container info(such as meta, priority) locally. So KoordRuntimeProxy should store pod/container info during RunPodSandbox/CreateContainer Stage. When StartContainer request comes, KoordRuntimeProxy can get pod/container info by containerID, and then call plugins with pod/container info.

With store, there would be pod/container info everytime KoordRuntimeProxy calls plugins, so there is no need for plugins to store pod/container info exceptionally, plugins can be designed as stateless.

Considering performance, store locates in memory and does not generate external io to disk.

Runtime Plugins

How to Register Plugins

All the plugin config files should be put to /etc/runtime/hookserver.d with .json suffix. You can register the plugin implemented by koordlet with RuntimeProxy:

1. touch /etc/runtime/hookserver.d/koordlet.json
2. Copy the following content into /etc/runtime/hookserver.d/koordlet.json

{
    "remote-endpoint": "/var/run/koordlet/koordlet.sock",
    "failure-policy": "Ignore",
    "runtime-hooks": [
        "PreRunPodSandbox",
        "PreCreateContainer",
        "PreStartContainer"
    ]
}

There are 3 fields involved:

• remote-endpoint: endpoint KoordRuntimeProxy talking with plugin, generated by plugin.
• failure-policy: policy when calling plugin fail, Fail or Ignore, default to Ignore.
• runtime-hooks: currently 7 hook points:
  1. PreRunPodSandbox
  2. PreCreateContainer
  3. PreStartContainer
  4. PostStartContainer
  5. PreUpdateContainerResources
  6. PostStopContainer
  7. PostStopPodSandbox

hook points with prefix 'Pre' means calling plugins before transferring request to contianerd(dockerd). hook points with prefix 'Post' means calling plugins after receiving response from containerd(dockerd). plugin provider can set any hook combinations to "runtime-hooks".

Protocols between KoordRuntimeProxy and Plugins

Protocols

Examples for Runtime Plugins

koordlet-runtime-plugin-design

Installation

Installing from sources

get sources: git clone https://github.com/koordinator-sh/koordinator.git

build: cd koordinator; make build-koord-runtime-proxy

Installing from packages

Download latest released package from: https://github.com/koordinator-sh/koordinator/releases

Setup Kubelet

Under containerd scenario, to make koord-runtime-proxy a proxy between kubelet and containerd, kubelet parameters should be altered as shown below:

kubelet <other options> --container-runtime=remote --container-runtime-endpoint=unix:///var/run/koord-runtimeproxy/runtimeproxy.sock

Under docker scenario, to make koord-runtime-proxy a proxy between kubelet and dockerd, kubelet parameters should be altered as shown below:

kubelet <other options> --docker-endpoint=unix:///var/run/koord-runtimeproxy/runtimeproxy.sock

Setup KoordRuntimeProxy

Firstly, please make sure your runtime backend is containerd or dockerd.

Under containerd scenario, koord-runtime-proxy can be setup with command:

koord-runtime-proxy --remote-runtime-service-endpoint=<runtime sockfile path>
    --remote-image-service-endpoint=<image sockfile path>

If containerd listening CRI request on default /var/run/koord-runtimeproxy/runtimeproxy.sock, koord-runtime-proxy can be setup by:

koord-runtime-proxy

Under docker scenario, koord-runtime-proxy should be setup with the additional parameter --backend-runtime-mode Docker, and without remote-image-service-endpoint:

koord-runtime-proxy --backend-runtime-mode=Docker --remote-runtime-service-endpoint=<runtime sockfile path>