Welcome to the new VergeOS docs! The legacy docs remain available until they're retired later this year.
Legacy VergeOS Docs
Ctrlk
Verge.ioGitHub repoSupport
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Kubernetes Integration

VergeOS provides native Kubernetes integration via a CSI storage driver and Cloud Controller Manager, giving clusters on VergeOS VMs persistent storage, node lifecycle management, and load balancers.

Overview

VergeOS provides native Kubernetes integration through a CSI storage driver and a Cloud Controller Manager (CCM). These components connect any Kubernetes cluster running on VergeOS VMs — whether provisioned through Rancher, kubeadm, or any other method — to the underlying VergeOS platform.

Component
Purpose

CSI Driver

Persistent storage for Kubernetes pods via VergeOS vSAN

Cloud Controller Manager

Node lifecycle and load balancer integration

Prerequisites

  • A Kubernetes cluster running on VergeOS VMs (RKE2, K3s, kubeadm, etc.)

  • A VergeOS API key (generated in User Settings)

  • kubectl and helm CLI tools

  • Access to the cluster's kubeconfig

Self-Signed Certificates

If the VergeOS environment uses a self-signed certificate, set vergeos.verifySSL=false when installing the CSI driver and CCM Helm charts. This is the default, but worth noting if we change it later.

CSI Driver

The Container Storage Interface (CSI) driver lets Kubernetes pods request and mount VergeOS storage as persistent volumes. It delegates storage operations directly to the VergeOS API, using the vSAN for what it was built for — deduplication, multi-tier placement, and distributed redundancy.

Storage Backends

The CSI driver supports two backends, both served by a single Go binary:

Backend
Access Mode
Description

NAS

ReadWriteMany

EXT4 volumes on VergeOS NAS services, exposed over NFS

Block

ReadWriteOnce

VM drives hotplugged to VergeOS VMs via the vSAN

Why Not Longhorn?

Longhorn runs its own replicated storage engine inside Kubernetes, layering replication, snapshots, and scheduling on top of the hypervisor. On VergeOS, the native CSI driver is a better fit:

  • No double replication — the vSAN's distributed mirror architecture already provides data redundancy

  • Inline deduplication — vSAN dedup is global across the cluster; Longhorn volumes are opaque blobs that can't participate

  • Multi-tier placement — volumes land on the correct vSAN tier (NVMe, SSD, HDD) based on the StorageClass

  • Unified management — volumes appear in the VergeOS UI alongside VMs, snapshots, and NAS shares

Block Storage Pool VM

For block storage, create an empty VM in VergeOS named k8spool. It never needs to boot — it just holds idle block drives. The VM's ID is passed to the Helm chart during installation.

Installation

The CSI driver is distributed as a Helm chart from the verge-io Helm repository and can be installed via the Helm CLI or through the Rancher Apps UI on downstream clusters. See Documentation and Resources below for chart details, Helm values, and configuration options.

Downstream Cluster

When installing through the Rancher Apps UI, the Helm repository must be added on the downstream cluster, not the Rancher management cluster. ClusterRepos do not propagate from the management cluster.

Cloud Controller Manager

The Cloud Controller Manager (CCM) is the standard Kubernetes cloud provider interface for VergeOS. It handles node lifecycle management and load balancer provisioning.

Features

  • Node Management — Populates Kubernetes node metadata (provider ID, instance type, internal IPs) from VergeOS VMs. Detects VM existence and power state for node lifecycle management.

  • Load Balancing — Provisions VergeOS VNet NAT/translate rules for type: LoadBalancer Services. Allocates IPs from a configurable pool and maps service ports to node ports automatically.

How Node Lifecycle Works

The CCM implements the Kubernetes InstancesV2 interface:

  1. InstanceMetadata — Resolves a Kubernetes node to a VergeOS VM (by provider ID or name), then returns the provider ID (vergeos://<vm-id>), instance type (<cpu>cpu-<ram>mb), and internal IP addresses from VM NICs

  2. InstanceExists — Checks whether the backing VM still exists in VergeOS

  3. InstanceShutdown — Reports whether the VM is powered off

How Load Balancing Works

The CCM implements the Kubernetes LoadBalancer interface using VergeOS VNet rules:

  1. EnsureLoadBalancer — Allocates an IP from the pool, then creates a VNet NAT rule per service port. Rules translate <allocated-ip>:<port> to <node-ips>:<nodePort>. Calls ApplyRules to activate.

  2. UpdateLoadBalancer — Updates the target IPs on existing rules when nodes change

  3. EnsureLoadBalancerDeleted — Deletes all VNet rules for the service and applies the change

Installation

The CCM is distributed as a Helm chart alongside the CSI driver. Install via the Helm CLI or through the Rancher Apps UI on the downstream cluster. See Documentation and Resources below for chart details and configuration options.

Cluster Autoscaler

The Kubernetes Cluster Autoscaler works with VergeOS-backed clusters provisioned through Rancher. It automatically adjusts the number of nodes in a pool based on pending pod resource requests — scaling up when pods can't be scheduled and scaling down when nodes are underutilized. See Documentation and Resources below for the upstream project link.

How It Works

The autoscaler uses Rancher's API to manage node pools:

  1. Scale up — When pods are pending due to insufficient resources, the autoscaler increases the node pool size. Rancher then uses the Docker Machine driver to provision new VergeOS VMs.

  2. Scale down — When nodes are underutilized for a configurable period, the autoscaler cordons, drains, and removes them. The driver deletes the backing VergeOS VMs.

Resource Requests

The autoscaler makes scaling decisions based on pod resource requests, not actual usage. Make sure workloads define requests in their pod specs for accurate scaling behavior.

Summary

VergeOS provides three Kubernetes components that work together with any cluster running on VergeOS VMs:

  • CSI Driver — persistent volumes via VergeOS NAS (ReadWriteMany) and vSAN block storage (ReadWriteOnce)

  • Cloud Controller Manager — node lifecycle management and LoadBalancer Services backed by VergeOS VNet rules

  • Cluster Autoscaler — automatic node pool scaling for Rancher-provisioned clusters

All components are distributed as Helm charts from the verge-io repository and can be installed via the Helm CLI or the Rancher Apps UI.

Next Steps

  • New to Rancher on VergeOS? See the Rancher Integration guide for cluster provisioning, template VM preparation, and the node driver setup.

  • Already have a cluster running? Add the Helm repository and explore the available charts:

  • Visit the project repositories below for chart values, configuration reference, and release notes.

Documentation and Resources

Repository
Description

csi-vergeos

Container Storage Interface driver

vergeos-cloud-controller-manager

Cloud Controller Manager

helm-charts

Helm chart repository

Kubernetes Cluster Autoscaler

Upstream cluster autoscaler

Support

If you encounter issues or have feature requests, please open an issue on the relevant GitHub repository:

PreviousInuvika Integration with VergeOS
NextLeostream Integration with VergeOS

Last updated

Was this helpful?