Skip to main content
Version: v0.7

Installation

Since v0.1.0 (alpha/beta), Koordinator requires Kubernetes version >= 1.18.

Koordinator need collect metrics from kubelet read-only port(default is disabled). you can get more info form here.

For the best experience, koordinator recommands linux kernel 4.19 or higher.

Install with helms

Koordinator can be simply installed by helm v3.5+, which is a simple command-line tool and you can get it from here.

# Firstly add koordinator charts repository if you haven't do this.
$ helm repo add koordinator-sh https://koordinator-sh.github.io/charts/

# [Optional]
$ helm repo update

# Install the latest version.
$ helm install koordinator koordinator-sh/koordinator --version 0.7.0

Upgrade with helm

# Firstly add koordinator charts repository if you haven't do this.
$ helm repo add koordinator-sh https://koordinator-sh.github.io/charts/

# [Optional]
$ helm repo update

# Upgrade the latest version.
$ helm upgrade koordinator koordinator-sh/koordinator --version 0.7.0 [--force]

Note that:

  1. Before upgrade, you must firstly read the Change Log to make sure that you have understand the breaking changes in the new version.
  2. If you want to drop the chart parameters you configured for the old release or set some new parameters, it is recommended to add --reset-values flag in helm upgrade command. Otherwise you should use --reuse-values flag to reuse the last release's values.

Optional: download charts manually

If you have problem with connecting to https://koordinator-sh.github.io/charts/ in production, you might need to download the chart from here manually and install or upgrade with it.

$ helm install/upgrade koordinator /PATH/TO/CHART

Install koord-runtime-proxy

koord-runtime-proxy acts as a proxy between kubelet and containerd(dockerd under dockershim scenario), which is designed to intercept CRI request, and apply some resource management policies, such as setting different cgroup parameters by pod priorities under hybrid workload orchestration scenario, applying new isolation policies for latest Linux kernel, CPU architecture, and etc.

1、Get binary

Download from github releases:

$ # select the version
$ wget https://github.com/koordinator-sh/koordinator/releases/download/v0.7.0/koord-runtime-proxy_0.7.0_linux_x86_64 -O koord-runtime-proxy
$ chmod +x koord-runtime-proxy

Or you can build from source:

$ git clone https://github.com/koordinator-sh/koordinator.git
$ cd koordinator
$ make build-koord-runtime-proxy

2、Setup koord-runtime-proxy

Firstly, please make sure your runtime backend is containerd or dockerd.

Under containerd scenario, if your containerd listening CRI request on default /var/run/containerd/containerd.sock, koord-runtime-proxy can be setup by(no need to set any parameters):

koord-runtime-proxy

Or koord-runtime-proxy can be setup with command:

koord-runtime-proxy \
--remote-runtime-service-endpoint=<runtime socketFile path> \
--remote-image-service-endpoint=<image socketFile path>

Under docker scenario, koord-runtime-proxy should be setup with the additional parameter --backend-runtime-mode Docker, and without remote-image-service-endpoint:

koord-runtime-proxy \
--backend-runtime-mode=Docker \
--remote-runtime-service-endpoint=<runtime socketFile path>

koord-runtime-proxy will listen on /var/run/koord-runtimeproxy/runtimeproxy.sock.

3、Setup Kubelet

To make koord-runtime-proxy a proxy between kubelet and containerd, kubelet parameters should be altered as shown below:

kubelet <other options> \
--container-runtime=remote \
--container-runtime-endpoint=/var/run/koord-runtimeproxy/runtimeproxy.sock

Under docker scenario, to make koord-runtime-proxy a proxy between kubelet and dockerd, kubelet parameters should be altered as shown below:

kubelet <other options> --docker-endpoint=/var/run/koord-runtimeproxy/runtimeproxy.sock

Options

Note that installing this chart directly means it will use the default template values for Koordinator.

You may have to set your specific configurations if it is deployed into a production cluster, or you want to configure feature-gates.

Optional: chart parameters

The following table lists the configurable parameters of the chart and their default values.

ParameterDescriptionDefault
featureGatesFeature gates for Koordinator, empty string means all by default
installation.namespacenamespace for Koordinator installationkoordinator-system
installation.createNamespaceWhether to create the installation.namespacetrue
imageRepositoryHostImage repository hostghcr.io
manager.log.levelLog level that koord-manager printed4
manager.replicasReplicas of koord-manager deployment2
manager.image.repositoryRepository for koord-manager imagekoordinatorsh/koord-manager
manager.image.tagTag for koord-manager image0.7.0
manager.resources.limits.cpuCPU resource limit of koord-manager container1000m
manager.resources.limits.memoryMemory resource limit of koord-manager container1Gi
manager.resources.requests.cpuCPU resource request of koord-manager container500m
manager.resources.requests.memoryMemory resource request of koord-manager container256Mi
manager.metrics.portPort of metrics served8080
manager.webhook.portPort of webhook served9443
manager.nodeAffinityNode affinity policy for koord-manager pod{}
manager.nodeSelectorNode labels for koord-manager pod{}
manager.tolerationsTolerations for koord-manager pod[]
manager.resyncPeriodResync period of informer koord-manager, defaults no resync0
manager.hostNetworkWhether koord-manager pod should run with hostnetworkfalse
scheduler.log.levelLog level that koord-scheduler printed4
scheduler.replicasReplicas of koord-scheduler deployment2
scheduler.image.repositoryRepository for koord-scheduler imagekoordinatorsh/koord-scheduler
scheduler.image.tagTag for koord-scheduler image0.7.0
scheduler.resources.limits.cpuCPU resource limit of koord-scheduler container1000m
scheduler.resources.limits.memoryMemory resource limit of koord-scheduler container1Gi
scheduler.resources.requests.cpuCPU resource request of koord-scheduler container500m
scheduler.resources.requests.memoryMemory resource request of koord-scheduler container256Mi
scheduler.portPort of metrics served10251
scheduler.nodeAffinityNode affinity policy for koord-scheduler pod{}
scheduler.nodeSelectorNode labels for koord-scheduler pod{}
scheduler.tolerationsTolerations for koord-scheduler pod[]
scheduler.hostNetworkWhether koord-scheduler pod should run with hostnetworkfalse
koordlet.log.levelLog level that koordlet printed4
koordlet.image.repositoryRepository for koordlet imagekoordinatorsh/koordlet
koordlet.image.tagTag for koordlet image0.7.0
koordlet.resources.limits.cpuCPU resource limit of koordlet container500m
koordlet.resources.limits.memoryMemory resource limit of koordlet container256Mi
koordlet.resources.requests.cpuCPU resource request of koordlet container0
koordlet.resources.requests.memoryMemory resource request of koordlet container0
webhookConfiguration.failurePolicy.podsThe failurePolicy for pods in mutating webhook configurationIgnore
webhookConfiguration.timeoutSecondsThe timeoutSeconds for all webhook configuration30
crds.managedKoordinator will not install CRDs with chart if this is falsetrue
imagePullSecretsThe list of image pull secrets for koordinator imagefalse

Specify each parameter using the --set key=value[,key=value] argument to helm install or helm upgrade.

Optional: feature-gate

Feature-gate controls some influential features in Koordinator:

NameDescriptionDefaultEffect (if closed)
PodMutatingWebhookWhether to open a mutating webhook for Pod createtrueDon't inject koordinator.sh/qosClass, koordinator.sh/priority and don't replace koordinator extend resources ad so on
PodValidatingWebhookWhether to open a validating webhook for Pod create/updatetrueIt is possible to create some Pods that do not conform to the Koordinator specification, causing some unpredictable problems

If you want to configure the feature-gate, just set the parameter when install or upgrade. Such as:

$ helm install koordinator https://... --set featureGates="PodMutatingWebhook=true\,PodValidatingWebhook=true"

If you want to enable all feature-gates, set the parameter as featureGates=AllAlpha=true.

Optional: the local image for China

If you are in China and have problem to pull image from official DockerHub, you can use the registry hosted on Alibaba Cloud:

$ helm install koordinator https://... --set imageRepositoryHost=registry.cn-beijing.aliyuncs.com

Best Practices

Installation parameters for AWS EKS

When using a custom CNI (such as Weave or Calico) on EKS, the webhook cannot be reached by default. This happens because the control plane cannot be configured to run on a custom CNI on EKS, so the CNIs differ between control plane and worker nodes.

To address this, the webhook can be run in the host network so it can be reached, by setting --set manager.hostNetwork=true when use helm install or upgrade.

Uninstall

Note that this will lead to all resources created by Koordinator, including webhook configurations, services, namespace, CRDs and CR instances managed by Koordinator controller, to be deleted!

Please do this ONLY when you fully understand the consequence.

To uninstall koordinator if it is installed with helm charts:

$ helm uninstall koordinator
release "koordinator" uninstalled