티스토리 뷰

Kubernetes

[7주차] RKE2 & Cluster API

넷오빠 2026. 2. 22. 21:36
이 포스팅은 Cloud@net 가시다님의 스터디 강의 내용을 요약한 자료입니다.
무단 베포를 금지합니다.

 

 RKE2 소개 

소개 : Rancher 에서 개발한 엔터프라이즈 K8S 배포판, 미국 연방 정부 부문의 보안 및 규정 준수 - Docs

  • 운영자의 개입을 최소화하면서 클러스터가 CIS Kubernetes Benchmark v1.7 또는 v1.8 을 통과할 수 있도록 기본 설정 및 구성 옵션을 제공합니다 .
  • RKE2는 컨트롤 플레인 구성 요소를 kubelet이 관리하는 정적 Pod로 실행합니다. 내장 컨테이너 런타임은 containerd입니다.
  • 이 배포판은 독립 실행형으로 실행되거나 Rancher에 통합되어 실행될 수 있습니다.

아키텍처 : https://docs.rke2.io/architecture

 

Anatomy of a Next Generation Kubernetes Distribution | RKE2

Architecture Overview

docs.rke2.io

 

 

  • RKE2는 경량 Kubernetes 배포판인 K3s를 개발하고 유지 관리하면서 얻은 교훈을 바탕으로, K3s의 사용 편의성을 유지하면서도 엔터프라이즈 환경에 적합한 배포판을 구축한 제품입니다.
  • 즉, RKE2는 가장 간단하게 말하면 Kubernetes 클러스터에 참여할 모든 노드에 설치 및 구성해야 하는 단일 바이너리 파일입니다.
  • 일단 시작되면 RKE2는 각 노드의 역할에 맞는 에이전트를 부트스트랩하고 관리하는 동시에 네트워크에서 필요한 콘텐츠를 가져올 수 있습니다

 

 RKE2 구성요소 

 

  • K3S
    • Helm Controller : GitOps 스타일 경량 배포기, A simple way to manage helm charts with Custom Resource Definitions in k8s - Docs
      • RKE2 클러스터가 올라올 때 필수 Helm 차트를 자동으로 설치·유지하는 내장 배포 컨트롤러.
      • manifests 디렉토리 감시 방식 : /var/lib/rancher/rke2/server/manifests 디렉토리에 (HelmChart) YAML 파일만 떨어뜨리면 Helm Controller에 의해 자동으로 Helm chart를 (다운로드)/설치/업데이트.
tree /var/lib/rancher/rke2/server/manifests
├── rke2-canal-config.yaml
├── rke2-canal.yaml
├── rke2-coredns-config.yaml
├── rke2-coredns.yaml
├── rke2-metrics-server.yaml
└── rke2-runtimeclasses.yaml

 

용도 비교 : Helm Ctrl 부트스트랩 (부팅 시 필수 애드온 자동 설치) vs Argo CD GitOps (애플리케이션 배포).

 

Kubernetes 구성요소 기능 요약

 

1-1. API Server (kube-apiserver)

  • 쿠버네티스의 관문(Front Door) 역할
  • kubectl, Controller, Scheduler 등 모든 컴포넌트가 API Server를 통해 통신
  • 클러스터 상태 조회/변경 요청을 받아서 검증 후 저장(etcd 반영)
  • 인증(Authentication), 인가(Authorization), Admission(정책 적용)도 담당

 

1-2. Controller Manager (kube-controller-manager)

  • 클러스터가 원하는 상태(Desired State) 를 유지하도록 계속 감시/조정
  • 예:
    • Deployment Controller → 파드 개수 맞추기
    • Node Controller → 노드 상태 확인
    • ReplicaSet Controller → 복제본 유지
    • Job Controller → 배치 작업 완료 관리

1-3. Scheduler (kube-scheduler)

  • 새로 생성될 Pod를 어느 노드에 배치할지 결정
  • CPU/메모리 여유, taint/toleration, affinity/anti-affinity, topology 등을 고려
  • 실제 실행은 kubelet이 하고, scheduler는 배치 결정만 수행

 

1-4. Proxy (kube-proxy)

  • 각 노드에서 Service 네트워킹 구현
  • Service IP(ClusterIP)로 들어온 트래픽을 실제 Pod로 전달
  • 보통 iptables 또는 ipvs 기반 룰 생성
  • 로드밸런싱(간단한 분산) 역할도 수행

 

1-5. Kubelet (kubelet)

  • 노드 에이전트 (각 워커 노드에 실행)
  • API Server에서 할당된 PodSpec을 보고 실제로 컨테이너 실행 상태 유지
  • container runtime(containerd 등)와 연동하여 컨테이너 생성/삭제/헬스체크
  • Pod 상태를 API Server에 보고

 

1-6. etcd

  • 쿠버네티스의 분산 Key-Value 저장소
  • 클러스터의 모든 상태 정보(리소스 정의, 설정, 메타데이터 등)를 저장
  • Control Plane의 “단일 진실 공급원(Source of Truth)”
  • 고가용성(HA) 구성 시 매우 중요하며 백업/복구 전략 필수

 

1-7. runc

  • OCI(Open Container Initiative) 표준 기반의 저수준 컨테이너 실행기
  • 실제로는 컨테이너 프로세스를 Linux namespace/cgroup으로 격리해 실행
  • 사용자가 직접 다루기보다는 보통 containerd가 내부적으로 사용

 

1-8. containerd / CRI

containerd

  • 고수준 컨테이너 런타임
  • 이미지 pull, 스냅샷, 컨테이너 lifecycle 관리 담당
  • 내부적으로 runc를 사용해 실제 컨테이너 실행

1-9. CRI (Container Runtime Interface)

  • 쿠버네티스가 런타임과 통신하기 위한 표준 인터페이스
  • kubelet은 CRI를 통해 containerd 같은 런타임을 제어
  • 덕분에 런타임 교체/호환성이 좋아짐

 

1-10. CNI (Container Network Interface)

  • Pod 네트워크를 구성하기 위한 플러그인 표준
  • Pod에 IP 부여, 네트워크 연결, 라우팅/오버레이, 정책 연계 등을 담당
  • “쿠버네티스 기본 기능”이 아니라 CNI 플러그인이 실제 네트워크를 구현함

 

1-11. CoreDNS

  • 클러스터 내부 DNS 서버
  • Service 이름(my-service.default.svc.cluster.local)을 IP로 변환
  • Pod가 다른 서비스 찾을 때 핵심 역할
  • 필요 시 외부 DNS 포워딩도 가능

 

1-12. ngress NGINX Controller / Traefik

둘 다 외부 HTTP/HTTPS 요청을 클러스터 내부 서비스로 라우팅하는 Ingress Controller입니다.

Ingress NGINX Controller

  • NGINX 기반, 가장 보편적
  • 기능/레퍼런스 많고 운영 사례 풍부
  • 세밀한 튜닝과 annotation 기반 설정이 강점

 

Process Lifecycle

 

Content Boostrap

  • RKE2 런타임 이미지로 부터 sources binaries and manifests 추출하여, 서버 노드와 에이전트 노드를 모두 실행. 
  • (참고) rke2-runtime 컨테이너 내부에 파일 확인 : 바이너리, HelmChart
crictl images
IMAGE                                           TAG                            IMAGE ID            SIZE
docker.io/rancher/rke2-runtime                  v1.34.3-rke2r3                 30afde048693a       91.6MB

tree
├── bin
│   ├── containerd
│   ├── containerd-shim-runc-v2
│   ├── crictl
│   ├── ctr
│   ├── kubectl
│   ├── kubelet
│   └── runc
├── charts
│   ├── harvester-cloud-provider.yaml
│   ├── harvester-csi-driver.yaml
│   ├── rancher-vsphere-cpi.yaml
│   ├── rancher-vsphere-csi.yaml
│   ├── rke2-calico-crd.yaml
│   ├── rke2-calico.yaml
│   ├── rke2-canal.yaml
│   ├── rke2-cilium.yaml
│   ├── rke2-coredns.yaml
│   ├── rke2-flannel.yaml
│   ├── rke2-ingress-nginx.yaml
│   ├── rke2-metrics-server.yaml
│   ├── rke2-multus.yaml
│   ├── rke2-runtimeclasses.yaml
│   ├── rke2-snapshot-controller-crd.yaml
│   ├── rke2-snapshot-controller.yaml
│   ├── rke2-snapshot-validation-webhook.yaml
│   ├── rke2-traefik-crd.yaml
│   └── rke2-traefik.yaml

 

 

  • RKE2 런타임 이미지는 /var/lib/rancher/rke2/agent/images/*.tar 에서 스캔하며(태그는 rke2 --version 출력), 이를 찾을 수 없는 경우 DockerHub 에서 찾습니다.

        (참고) /var/lib/rancher/rke2/agent/images/ 에 runtime-image.txt 파일 확인

tree /var/lib/rancher/rke2/agent/images/
├── etcd-image.txt
├── kube-apiserver-image.txt
├── kube-controller-manager-image.txt
├── kube-proxy-image.txt
├── kube-scheduler-image.txt
└── runtime-image.txt

cat /var/lib/rancher/rke2/agent/images/runtime-image.txt 
index.docker.io/rancher/rke2-runtime:v1.34.3-rke2r3
  • 그런 다음 RKE2는 런타임 이미지에서 /bin/을 추출하여 /var/lib/rancher/rke2/data/${RKE2_DATA_KEY}/bin 으로 평탄화합니다.

 

  • 여기서 ${RKE2_DATA_KEY}는 이미지를 식별(버전)하는 고유 문자열을 나타냅니다. 

(참고) /var/lib/rancher/rke2/data/ 확인

tree /var/lib/rancher/rke2/data/
└── v1.34.3-rke2r3-5b8349de68df
    ├── bin
    │   ├── containerd
    │   ├── containerd-shim-runc-v2
    │   ├── crictl
    │   ├── ctr
    │   ├── kubectl
    │   ├── kubelet
    │   └── runc
    └── charts
        ├── harvester-cloud-provider.yaml
        ├── harvester-csi-driver.yaml
        ├── rancher-vsphere-cpi.yaml
        ├── rancher-vsphere-csi.yaml
        ├── rke2-calico-crd.yaml
        ├── rke2-calico.yaml
        ├── rke2-canal.yaml
        ├── rke2-cilium.yaml
        ├── rke2-coredns.yaml
        ├── rke2-flannel.yaml
        ├── rke2-ingress-nginx.yaml
        ├── rke2-metrics-server.yaml
        ├── rke2-multus.yaml
        ├── rke2-runtimeclasses.yaml
        ├── rke2-snapshot-controller-crd.yaml
        ├── rke2-snapshot-controller.yaml
        ├── rke2-snapshot-validation-webhook.yaml
        ├── rke2-traefik-crd.yaml
        └── rke2-traefik.yaml

 

 

  • RKE2가 예상대로 작동하려면 런타임 이미지는 최소한 다음을 제공해야 합니다.
    • containerd (CRI)
    • containerd-shim (shims wrap runc tasks and do not stop when containerd does)
    • containerd-shim-runc-v1
    • containerd-shim-runc-v2
    • kubelet (쿠버네티스 노드 에이전트)
    • runc (OCI 런타임)
  • 런타임 이미지에는 다음과 같은 운영 도구도 포함되어 있습니다.
    • **ctr**containerd (저수준 유지보수 및 점검)
    • crictl (낮은 수준의 CRI 유지 관리 및 점검)
    • kubectl (쿠버네티스 클러스터 유지 관리 및 점검)
    • **socat**containerd ( 포트 포워딩 에 필요함 )
  • 바이너리 파일 압축 해제가 완료되면 RKE2는 이미지에서 차트를 추출하여 해당 /var/lib/rancher/rke2/server/manifests디렉터리에 저장합니다.
tree /var/lib/rancher/rke2/server/manifests
├── rke2-canal-config.yaml
├── rke2-canal.yaml
├── rke2-coredns-config.yaml
├── rke2-coredns.yaml
├── rke2-metrics-server.yaml
└── rke2-runtimeclasses.yaml

 

 

 

Initialize Server

임베디드 K3s 엔진 서버에는 특수 에이전트 프로세스가 포함되어 있어 노드 컨테이너 런타임 node container runtime 이 시작될 때까지 다음 시작이 연기됩니다.

(참고) /var/lib/rancher/rke2/agent/pod-manifests/ static pod

ls -ltr /var/lib/rancher/rke2/agent/pod-manifests/
total 32
-rw-r--r--. 1 root root 3279 Feb 14 16:32 etcd.yaml
-rw-r--r--. 1 root root 2325 Feb 14 16:32 kube-proxy.yaml
-rw-r--r--. 1 root root 9337 Feb 14 16:33 kube-apiserver.yaml
-rw-r--r--. 1 root root 3724 Feb 14 16:33 kube-scheduler.yaml
-rw-r--r--. 1 root root 6325 Feb 14 16:33 kube-controller-manager.yaml

 

Prepare Components

  • kube-apiserver
    • Kube-apiserver 이미지가 아직 존재하지 않는 경우, Goroutine을 회전시켜 etcd를 기다린 다음 /var/lib/rancher/rke2/agent/pod-manifests/에 정적 포드 정의 static pod definition 를 작성합니다.
  • kube-controller-manager
    • kube-controller-manager 이미지가 아직 존재하지 않는 경우, goroutine을 회전시켜 kube-apiserver를 기다린 다음 /var/lib/rancher/rke2/agent/pod-manifests/에 정적 포드 정의를 작성합니다.
  • kube-scheduler
    • kube-scheduler 이미지가 아직 없는 경우, 고루틴을 회전시켜 kube-apiserver를 기다린 다음 /var/lib/rancher/rke2/agent/pod-manifests/에 정적 포드 정의를 작성합니다.

 

 

Start Cluster

Spin up an HTTP server in a goroutine to listen for other cluster servers/agents then initialize/join the cluster.

  • etcd
    • 이미 존재하지 않는 경우 etcd 이미지를 가져와서 Goroutine을 회전시켜 kubelet을 기다린 다음 /var/lib/rancher/rke2/agent/pod-manifests/에 정적 포드 정의를 작성합니다.
  • helm-controller
    • kube-apiserver가 준비될 때까지 기다린 후 Goroutine을 회전시켜 내장된 헬름 컨트롤러 embedded helm-controller 를 시작합니다.

rke2-server 프로세스

#
pstree -al
  ├─rke2
  │   ├─containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml
  │   │   └─12*[{containerd}]
  │   ├─kubelet --volume-plugin-dir=/var/lib/kubelet/volumeplugins --file-check-frequency=5s --sync-frequency=30s --config-dir=/var/lib/rancher/rke2/agent/etc/kubelet.conf.d --containerd=/run/k3s/containerd/containerd.sock --hostname-override=k8s-node1 --kubeconfig=/var/lib/rancher/rke2/agent/kubelet.kubeconfig --node-ip=192.168.10.11 --node-labels= --read-only-port=0
  │   │   └─14*[{kubelet}]
  │   └─12*[{rke2}]

systemctl status rke2-server.service --no-pager
● rke2-server.service - Rancher Kubernetes Engine v2 (server)
     Loaded: loaded (/usr/lib/systemd/system/rke2-server.service; enabled; preset: disabled)
     Active: active (running) since Sat 2026-02-14 16:33:49 KST; 1h 37min ago
       Docs: https://github.com/rancher/rke2#readme
    Process: 6346 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 6347 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 6348 (rke2)
      Tasks: 146
     Memory: 1.1G
        CPU: 5min 15.217s
     CGroup: /system.slice/rke2-server.service
             ├─6348 "/usr/bin/rke2 server"
             ├─6367 containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml
             ├─6426 kubelet --volume-plugin-dir=/var/lib/kubelet/volumeplugins --file-check-frequency=5s --sync-frequency…
             ├─6483 /var/lib/rancher/rke2/data/v1.34.3-rke2r3-5b8349de68df/bin/containerd-shim-runc-v2 -namespace k8s.io …
             ├─6484 /var/lib/rancher/rke2/data/v1.34.3-rke2r3-5b8349de68df/bin/containerd-shim-runc-v2 -namespace k8s.io …
             ...

# systemd unit 파일 확인
cat /usr/lib/systemd/system/rke2-server.service
[Unit]
Description=Rancher Kubernetes Engine v2 (server)
Documentation=https://github.com/rancher/rke2#readme
Wants=network-online.target
After=network-online.target
Conflicts=rke2-agent.service

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
EnvironmentFile=-/etc/default/%N
EnvironmentFile=-/etc/sysconfig/%N
EnvironmentFile=-/usr/lib/systemd/system/%N.env
KillMode=process   # "나만 죽이고, 내가 띄운 애들까지는 systemd가 건드리지 마" - RKE2, containerd 같은 애들은 내부적으로 여러 자식 프로세스 shim, runc, container 프로세스를 관리함. 물론 완전 무중단은 아니지만, 충격 최소화
Delegate=yes       # "cgroup(리소스 제어)는 내가 직접 관리할게" - cgroup(리눅스 리소스 그룹) 관리를 systemd가 아니라 자기 자신이 직접 하도록 위임(delegate) 하겠다는 의미.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/bin/rke2 server
ExecStopPost=-/bin/sh -c "systemd-cgls /system.slice/%n | grep -Eo '[0-9]+ (containerd|kubelet)' | awk '{print $1}' | xargs -r kill"

 

 

Initialize Agent

  • kubelet : Spawn and supervise the kubelet process.
  • kubelet이 종료되면 rke2가 재시작을 시도합니다. kubelet이 작동하면 사용 가능한 모든 정적 포드가 시작됩니다.
  • 서버의 경우, etcd와 kube-apiserver가 연속적으로 시작되어 정적 포드를 통해 시작된 나머지 구성 요소들이 kube-apiserver에 연결되어 처리를 시작할 수 있게 됩니다.

Server Charts

  • On server nodes, the helm-controller can now apply to the cluster any charts found /var/lib/rancher/rke2/server/manifests.
  • rke2*-**canal.yaml or rke2-cilium.yaml or rke2-calico.yaml or rke2-flannel.**yaml or rke2-**multus.*yaml (daemonset, bootstrap)
  • rke2-coredns.yaml (deployment, bootstrap)
  • rke2-ingress-nginx.yaml and/or rke2-traefik.yaml and rke2-traefik-crd.yaml (deployment)
  • rke2-metrics-server.yaml (deployment)
  • rke2-runtimeclasses.yaml (deployment)
  • rke2-snapshot-controller-crd.yaml, rke2-snapshot-controller.yaml and rke2-snapshot-validation-webhook.yaml (deployment)
  • Daemon Process : 이제 RKE2 프로세스는 SIGTERM 또는 SIGKILL을 받거나 컨테이너 프로세스가 종료될 때까지 무기한으로 실행됨.

 

서버 노드 부팅 시 과정

출처 : https://sirzzang.github.io/kubernetes/Kubernetes-RKE2-00/

 

 

에이전트 노드 부팅 시 과정

 

 

 RKE2 실습 

실습 환경 배포

 

Vagrantfile : Rocky Linux 9 

# Base Image  https://portal.cloud.hashicorp.com/vagrant/discover/bento/rockylinux-9
BOX_IMAGE = "bento/rockylinux-9" # "bento/rockylinux-10.0"
BOX_VERSION = "202510.26.0"
N = 2 # max number of Node

Vagrant.configure("2") do |config|

# Nodes 
  (1..N).each do |i|
    config.vm.define "k8s-node#{i}" do |subconfig|
      subconfig.vm.box = BOX_IMAGE
      subconfig.vm.box_version = BOX_VERSION
      subconfig.vm.provider "virtualbox" do |vb|
        vb.customize ["modifyvm", :id, "--groups", "/RKE2-Lab"]
        vb.customize ["modifyvm", :id, "--nicpromisc2", "allow-all"]
        vb.name = "k8s-node#{i}"
        vb.cpus = 4
        vb.memory = 4096
        vb.linked_clone = true
      end
      subconfig.vm.host_name = "k8s-node#{i}"
      subconfig.vm.network "private_network", ip: "192.168.10.1#{i}"
      subconfig.vm.network "forwarded_port", guest: 22, host: "6000#{i}", auto_correct: true, id: "ssh"
      subconfig.vm.synced_folder "./", "/vagrant", disabled: true
      subconfig.vm.provision "shell", path: "init_cfg.sh" , args: [ N ]
    end
  end

end

 

 

init_cfg.sh 

#!/usr/bin/env bash

echo ">>>> Initial Config Start <<<<"


echo "[TASK 1] Change Timezone and Enable NTP"
timedatectl set-local-rtc 0
timedatectl set-timezone Asia/Seoul


echo "[TASK 2] Disable firewalld and selinux"
systemctl disable --now firewalld >/dev/null 2>&1
setenforce 0
sed -i 's/^SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config


echo "[TASK 3] Disable and turn off SWAP & Delete swap partitions"
swapoff -a
sed -i '/swap/d' /etc/fstab
sfdisk --delete /dev/sda 2 >/dev/null 2>&1
partprobe /dev/sda >/dev/null 2>&1


echo "[TASK 4] Config kernel & module"
cat << EOF > /etc/modules-load.d/k8s.conf
overlay
br_netfilter
#vxlan
EOF
modprobe overlay >/dev/null 2>&1
modprobe br_netfilter >/dev/null 2>&1
#modprobe vxlan >/dev/null 2>&1

cat << EOF >/etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
sysctl --system >/dev/null 2>&1


echo "[TASK 5] Setting Local DNS Using Hosts file"
sed -i '/^127\.0\.\(1\|2\)\.1/d' /etc/hosts
for (( i=1; i<=$1; i++  )); do echo "192.168.10.1$i k8s-node$i" >> /etc/hosts; done


echo "[TASK 6] Install Helm"
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | DESIRED_VERSION=v3.20.0 bash >/dev/null 2>&1


echo "[TASK 7] Setting SSHD"
cat << EOF >> /etc/ssh/sshd_config
PermitRootLogin yes
PasswordAuthentication yes
EOF
systemctl restart sshd >/dev/null 2>&1


echo "[TASK 8] Install packages"
dnf install -y conntrack python3-pip git >/dev/null 2>&1


echo "[TASK 9] NetworkManager to ignore calico/flannel related network interfaces"
# https://docs.rke2.io/known_issues#networkmanager
cat << EOF > /etc/NetworkManager/conf.d/k8s.conf
[keyfile]
unmanaged-devices=interface-name:flannel*;interface-name:cali*;interface-name:tunl*;interface-name:vxlan.calico;interface-name:vxlan-v6.calico;interface-name:wireguard.cali;interface-name:wg-v6.cali
EOF
systemctl reload NetworkManager


echo "[TASK 11] Install K9s"
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
wget -P /tmp https://github.com/derailed/k9s/releases/latest/download/k9s_linux_${CLI_ARCH}.tar.gz  >/dev/null 2>&1
tar -xzf /tmp/k9s_linux_${CLI_ARCH}.tar.gz -C /tmp
chown root:root /tmp/k9s
mv /tmp/k9s /usr/local/bin/
chmod +x /usr/local/bin/k9s


echo "[TASK 12] ETC"
echo "sudo su -" >> /home/vagrant/.bashrc


echo ">>>> Initial Config End <<<<"

 

mkdir k8s-rke2
cd k8s-rke2

curl -O https://raw.githubusercontent.com/gasida/vagrant-lab/refs/heads/main/k8s-rke2/Vagrantfile
curl -O https://raw.githubusercontent.com/gasida/vagrant-lab/refs/heads/main/k8s-rke2/init_cfg.sh

vagrant up
vagrant status

 

 

 

RKE2 서버 노드 설치 - Docs, Rocky Linux 9 사용

설치 스크립트

#!/bin/sh

set -e

if [ "${DEBUG}" = 1 ]; then
    set -x
fi

# Usage:
#   curl ... | ENV_VAR=... sh -
#       or
#   ENV_VAR=... ./install.sh
#

# Environment variables:
#
#   - INSTALL_RKE2_CHANNEL
#     Channel to use for fetching rke2 download URL.
#     Defaults to 'stable'.
#
#   - INSTALL_RKE2_METHOD
#     The installation method to use.
#     Default is on RPM-based systems is "rpm", all else "tar".
#
#   - INSTALL_RKE2_TYPE
#     Type of rke2 service. Can be either "server" or "agent".
#     Default is "server".
#
#   - INSTALL_RKE2_EXEC
#     This is an alias for INSTALL_RKE2_TYPE, included for compatibility with K3s.
#     If both are set, INSTALL_RKE2_TYPE is preferred.
#
#   - INSTALL_RKE2_VERSION
#     Version of rke2 to download.
#
#   - INSTALL_RKE2_RPM_RELEASE_VERSION
#     Version of the rke2 RPM release to install.
#     Format would be like "1.el7" or "2.el8"
#
#   - INSTALL_RKE2_TAR_PREFIX
#     Installation prefix when using the tar installation method.
#     Default is /usr/local, unless /usr/local is read-only or has a dedicated mount point,
#     in which case /opt/rke2 is used instead.
#
#   - INSTALL_RKE2_COMMIT
#     Commit of RKE2 to download from temporary cloud storage.
#     For developer & QA use only
#
#   - INSTALL_RKE2_AGENT_IMAGES_DIR
#     Installation path for airgap images when installing from CI commit
#     Default is /var/lib/rancher/rke2/agent/images
#
#   - INSTALL_RKE2_ARTIFACT_URL
#     URL prefix for RKE2 release artifacts.
#     Default is https://github.com/rancher/rke2/releases/download
#
#   - INSTALL_RKE2_ARTIFACT_PATH
#     If set, the install script will use the local path for sourcing the rke2.linux-$SUFFIX and sha256sum-$ARCH.txt files
#     rather than the downloading the files from the internet.
#     Default is not set.
#
#   - INSTALL_RKE2_SKIP_RELOAD
#     If set, the install script will skip reloading systemctl daemon after the tar has been extracted and systemd units
#     have been moved.
#     Default is not set.
#
#   - INSTALL_RKE2_SKIP_FAPOLICY
#     If set, the install script will skip adding fapolicy rules
#     Default is not set.


# info logs the given argument at info log level.
info() {
    echo "[INFO] " "$@"
}

# warn logs the given argument at warn log level.
warn() {
    echo "[WARN] " "$@" >&2
}

# fatal logs the given argument at fatal log level.
fatal() {
    echo "[ERROR] " "$@" >&2
    if [ -n "${SUFFIX}" ]; then
        echo "[ALT] Please visit 'https://github.com/rancher/rke2/releases' directly and download the latest rke2.${SUFFIX}.tar.gz" >&2
    fi
    exit 1
}

# check_target_mountpoint return success if the target directory is on a dedicated mount point
check_target_mountpoint() {
    mountpoint -q "${INSTALL_RKE2_TAR_PREFIX}"
}

# check_target_ro returns success if the target directory is read-only
check_target_ro() {
    touch "${INSTALL_RKE2_TAR_PREFIX}"/.rke2-ro-test && rm -rf "${INSTALL_RKE2_TAR_PREFIX}"/.rke2-ro-test
    test $? -ne 0
}


# setup_env defines needed environment variables.
setup_env() {
    STORAGE_URL="https://rke2-ci-builds.s3.dualstack.us-east-1.amazonaws.com"
    DEFAULT_TAR_PREFIX="/usr/local"
    # --- bail if we are not root ---
    if [ ! $(id -u) -eq 0 ]; then
        fatal "You need to be root to perform this install"
    fi

    # --- make sure artifact url prefix has a value
    if [ -z "${INSTALL_RKE2_ARTIFACT_URL}" ]; then
        INSTALL_RKE2_ARTIFACT_URL="https://github.com/rancher/rke2/releases/download"
    fi

    # --- make sure install channel has a value
    if [ -z "${INSTALL_RKE2_CHANNEL}" ]; then
        INSTALL_RKE2_CHANNEL="stable"
    fi

    # --- make sure install type has a value
    if [ -z "${INSTALL_RKE2_TYPE}" ]; then
        INSTALL_RKE2_TYPE="${INSTALL_RKE2_EXEC:-server}"
    fi

    # --- use rpm install method if available by default
    if [ -z "${INSTALL_RKE2_ARTIFACT_PATH}" ] && [ -z "${INSTALL_RKE2_COMMIT}" ] && [ -z "${INSTALL_RKE2_METHOD}" ] && command -v yum >/dev/null 2>&1; then
        INSTALL_RKE2_METHOD="rpm"
    fi

    # --- install tarball to /usr/local by default, except if /usr/local is on a separate partition or is read-only
    # --- in which case we go into /opt/rke2.
    if [ -z "${INSTALL_RKE2_TAR_PREFIX}" ]; then
        INSTALL_RKE2_TAR_PREFIX=${DEFAULT_TAR_PREFIX}
        if check_target_mountpoint || check_target_ro; then
            INSTALL_RKE2_TAR_PREFIX="/opt/rke2"
            warn "${DEFAULT_TAR_PREFIX} is read-only or a mount point; installing to ${INSTALL_RKE2_TAR_PREFIX}"
        fi
    fi

    if [ -z "${INSTALL_RKE2_AGENT_IMAGES_DIR}" ]; then
        INSTALL_RKE2_AGENT_IMAGES_DIR="/var/lib/rancher/rke2/agent/images"
    fi
}

# check_method_conflict will exit with an error if the user attempts to install
# via tar method on a host with existing RPMs.
check_method_conflict() {
     . /etc/os-release
    case ${INSTALL_RKE2_METHOD} in
    yum | rpm | dnf)
        if [ "${ID_LIKE%%[ ]*}" = "suse" ]; then
            if [ -f ${INSTALL_RKE2_TAR_PREFIX}/bin/rke2 ]; then
                fatal "Cannot perform ${INSTALL_RKE2_METHOD:-rpm} install on host with existing tarball installation - please run rke2-uninstall.sh first"
            fi
        fi
        return
        ;;
    *)
        if rpm -q rke2-common >/dev/null 2>&1; then
            fatal "Cannot perform ${INSTALL_RKE2_METHOD:-tar} install on host with existing RKE2 RPMs - please run rke2-uninstall.sh first"
        fi
        ;;
    esac
}

# setup_arch set arch and suffix,
# fatal if architecture not supported.
setup_arch() {
    case ${ARCH:=$(uname -m)} in
    x86_64|amd64)
        ARCH=amd64
        SUFFIX=$(uname -s | tr '[:upper:]' '[:lower:]')-${ARCH}
        ;;
    aarch64|arm64)
        ARCH=arm64
        SUFFIX=$(uname -s | tr '[:upper:]' '[:lower:]')-${ARCH}
        ;;
    s390x)
        ARCH=s390x
        SUFFIX=$(uname -s | tr '[:upper:]' '[:lower:]')-${ARCH}
        ;;
    *)
        fatal "unsupported architecture ${ARCH}"
        ;;
    esac
}

# verify_downloader verifies existence of
# network downloader executable.
verify_downloader() {
    cmd="$(command -v "${1}")"
    if [ -z "${cmd}" ]; then
        return 1
    fi
    if [ ! -x "${cmd}" ]; then
        return 1
    fi

    # Set verified executable as our downloader program and return success
    DOWNLOADER=${cmd}
    return 0
}

# verify_fapolicyd verifies existence of
# fapolicyd and fagenrules executables and make sure that fapolicyd service is running.
verify_fapolicyd() {
    cmd="$(command -v "fapolicyd")"
    if [ -z "${cmd}" ]; then
        return 1
    fi
    cmd="$(command -v "fagenrules")"
    if [ -z "${cmd}" ]; then
        return 1
    fi
    systemctl is-active --quiet fapolicyd || return 1

    return 0
}

# setup_tmp creates a temporary directory
# and cleans up when done.
setup_tmp() {
    TMP_DIR=$(mktemp -d -t rke2-install.XXXXXXXXXX)
    TMP_CHECKSUMS=${TMP_DIR}/rke2.checksums
    TMP_TARBALL=${TMP_DIR}/rke2.tarball
    TMP_AIRGAP_CHECKSUMS=${TMP_DIR}/rke2-images.checksums
    TMP_AIRGAP_TARBALL=${TMP_DIR}/rke2-images.tarball
    cleanup() {
        code=$?
        set +e
        trap - EXIT
        rm -rf "${TMP_DIR}"
        exit $code
    }
    trap cleanup INT EXIT
}

# --- use desired rke2 version if defined or find version from channel ---
get_release_version() {
    if [ -z "${INSTALL_RKE2_COMMIT}" ] && [ -z "${INSTALL_RKE2_VERSION}" ]; then
        info "finding release for channel ${INSTALL_RKE2_CHANNEL}"
        INSTALL_RKE2_CHANNEL_URL=${INSTALL_RKE2_CHANNEL_URL:-'https://update.rke2.io/v1-release/channels'}
        version_url="${INSTALL_RKE2_CHANNEL_URL}/${INSTALL_RKE2_CHANNEL}"
        case ${DOWNLOADER} in
        *curl)
            version=$(${DOWNLOADER} -w "%{url_effective}" -L -s -S "${version_url}" -o /dev/null | sed -e 's|.*/||')
            ;;
        *wget)
            version=$(${DOWNLOADER} -SqO /dev/null "${version_url}" 2>&1 | grep -i Location | sed -e 's|.*/||')
            ;;
        *)
            fatal "Unsupported downloader executable '${DOWNLOADER}'"
            ;;
        esac
        INSTALL_RKE2_VERSION="${version}"
    fi
}

# check_download performs a HEAD request to see if a file exists at a given url
check_download() {
    case ${DOWNLOADER} in
    *curl)
        curl -o "/dev/null" -fsLI -X HEAD "$1"
        ;;
    *wget)
        wget -q --spider "$1"
        ;;
    *)
        fatal "downloader executable not supported: '${DOWNLOADER}'"
        ;;
    esac
}

# download downloads a file from a url using either curl or wget
download() {
    if [ $# -ne 2 ]; then
        fatal "download needs exactly 2 arguments"
    fi

    case ${DOWNLOADER} in
    *curl)
        curl -o "$1" -fsSL "$2"
        ;;
    *wget)
        wget -qO "$1" "$2"
        ;;
    *)
        fatal "downloader executable not supported: '${DOWNLOADER}'"
        ;;
    esac

    # Abort if download command failed
    if [ $? -ne 0 ]; then
        fatal "download failed"
    fi
}

# download_checksums downloads hash from github url.
download_checksums() {
    version_urlsafe="$(echo ${INSTALL_RKE2_VERSION} | sed 's/\+/%2B/g')"
    if [ -n "${INSTALL_RKE2_COMMIT}" ]; then
        CHECKSUMS_URL=${STORAGE_URL}/rke2.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.gz.sha256sum
    else
        CHECKSUMS_URL=${INSTALL_RKE2_ARTIFACT_URL}/${version_urlsafe}/sha256sum-${ARCH}.txt
    fi
    info "downloading checksums at ${CHECKSUMS_URL}"
    download "${TMP_CHECKSUMS}" "${CHECKSUMS_URL}"
    CHECKSUM_EXPECTED=$(grep "rke2.${SUFFIX}.tar.gz" "${TMP_CHECKSUMS}" | awk '{print $1}')
}

# download_tarball downloads binary from github url.
download_tarball() {
    version_urlsafe="$(echo ${INSTALL_RKE2_VERSION} | sed 's/\+/%2B/g')"
    if [ -n "${INSTALL_RKE2_COMMIT}" ]; then
        TARBALL_URL=${STORAGE_URL}/rke2.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.gz
    else
        TARBALL_URL=${INSTALL_RKE2_ARTIFACT_URL}/${version_urlsafe}/rke2.${SUFFIX}.tar.gz
    fi
    info "downloading tarball at ${TARBALL_URL}"
    download "${TMP_TARBALL}" "${TARBALL_URL}"
}

# download_dev_rpm downloads dev rpm from remote repository
download_dev_rpm() {
    if [ -z "${INSTALL_RKE2_COMMIT}" ]; then
        fatal 'Development rpm requires a commit hash'
    fi

    DEST="${1}"
    RPM_FILE=$(basename "${DEST}")
    RPM_URL="${STORAGE_URL}/${RPM_FILE}"

    info "downloading rpm at ${RPM_URL}"
    download "${DEST}" "${RPM_URL}"
}

# stage_local_checksums stages the local checksum hash for validation.
stage_local_checksums() {
    info "staging local checksums from ${INSTALL_RKE2_ARTIFACT_PATH}/sha256sum-${ARCH}.txt"
    cp -f "${INSTALL_RKE2_ARTIFACT_PATH}/sha256sum-${ARCH}.txt" "${TMP_CHECKSUMS}"
    CHECKSUM_EXPECTED=$(grep "rke2.${SUFFIX}.tar.gz" "${TMP_CHECKSUMS}" | awk '{print $1}')
    if [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst" ]; then
        AIRGAP_CHECKSUM_EXPECTED=$(grep "rke2-images.${SUFFIX}.tar.zst" "${TMP_CHECKSUMS}" | awk '{print $1}')
    elif [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.gz" ]; then
        AIRGAP_CHECKSUM_EXPECTED=$(grep "rke2-images.${SUFFIX}.tar.gz" "${TMP_CHECKSUMS}" | awk '{print $1}')
    fi
}

# stage_local_tarball stages the local tarball.
stage_local_tarball() {
    info "staging tarball from ${INSTALL_RKE2_ARTIFACT_PATH}/rke2.${SUFFIX}.tar.gz"
    cp -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2.${SUFFIX}.tar.gz" "${TMP_TARBALL}"
}

# stage_local_airgap_tarball stages the local checksum hash for validation.
stage_local_airgap_tarball() {
    if [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst" ]; then
        info "staging zst airgap image tarball from ${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst"
        cp -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst" "${TMP_AIRGAP_TARBALL}"
        AIRGAP_TARBALL_FORMAT=zst
    elif [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.gz" ]; then
        info "staging gzip airgap image tarball from ${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.gz"
        cp -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.gz" "${TMP_AIRGAP_TARBALL}"
        AIRGAP_TARBALL_FORMAT=gz
    fi
}

# verify_tarball verifies the downloaded installer checksum.
verify_tarball() {
    info "verifying tarball"
    CHECKSUM_ACTUAL=$(sha256sum "${TMP_TARBALL}" | awk '{print $1}')
    if [ "${CHECKSUM_EXPECTED}" != "${CHECKSUM_ACTUAL}" ]; then
        fatal "download sha256 does not match ${CHECKSUM_EXPECTED}, got ${CHECKSUM_ACTUAL}"
    fi
}

# unpack_tarball extracts the tarball, correcting paths and moving systemd units as necessary
unpack_tarball() {
    info "unpacking tarball file to ${INSTALL_RKE2_TAR_PREFIX}"
    mkdir -p ${INSTALL_RKE2_TAR_PREFIX}
    tar xzf "${TMP_TARBALL}" -C "${INSTALL_RKE2_TAR_PREFIX}"
    if [ "${INSTALL_RKE2_TAR_PREFIX}" != "${DEFAULT_TAR_PREFIX}" ]; then
        info "updating tarball contents to reflect install path"
        sed -i "s|${DEFAULT_TAR_PREFIX}|${INSTALL_RKE2_TAR_PREFIX}|" ${INSTALL_RKE2_TAR_PREFIX}/lib/systemd/system/rke2-*.service ${INSTALL_RKE2_TAR_PREFIX}/bin/rke2-uninstall.sh
        info "moving systemd units to /etc/systemd/system"
        mv -f ${INSTALL_RKE2_TAR_PREFIX}/lib/systemd/system/rke2-*.service /etc/systemd/system/
        info "install complete; you may want to run:  export PATH=\$PATH:${INSTALL_RKE2_TAR_PREFIX}/bin"
    fi
}

# download_airgap_checksums downloads the checksum file for the airgap image tarball
# and prepares the checksum value for later validation.
download_airgap_checksums() {
    if [ -z "${INSTALL_RKE2_COMMIT}" ]; then
        return
    fi
    AIRGAP_CHECKSUMS_URL=${STORAGE_URL}/rke2-images.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.zst.sha256sum
    # try for zst first; if that fails use gz for older release branches
    if ! check_download "${AIRGAP_CHECKSUMS_URL}"; then
        AIRGAP_CHECKSUMS_URL=${STORAGE_URL}/rke2-images.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.gz.sha256sum
    fi
    info "downloading airgap checksums at ${AIRGAP_CHECKSUMS_URL}"
    download "${TMP_AIRGAP_CHECKSUMS}" "${AIRGAP_CHECKSUMS_URL}"
    AIRGAP_CHECKSUM_EXPECTED=$(grep "rke2-images.${SUFFIX}.tar" "${TMP_AIRGAP_CHECKSUMS}" | awk '{print $1}')
}

# download_airgap_tarball downloads the airgap image tarball.
download_airgap_tarball() {
    if [ -z "${INSTALL_RKE2_COMMIT}" ]; then
        return
    fi
    AIRGAP_TARBALL_URL=${STORAGE_URL}/rke2-images.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.zst
    # try for zst first; if that fails use gz for older release branches
    if ! check_download "${AIRGAP_TARBALL_URL}"; then
        AIRGAP_TARBALL_URL=${STORAGE_URL}/rke2-images.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.gz
    fi
    info "downloading airgap tarball at ${AIRGAP_TARBALL_URL}"
    download "${TMP_AIRGAP_TARBALL}" "${AIRGAP_TARBALL_URL}"
}

# verify_airgap_tarball compares the airgap image tarball checksum to the value
# calculated by CI when the file was uploaded.
verify_airgap_tarball() {
    if [ -z "${AIRGAP_CHECKSUM_EXPECTED}" ]; then
        return
    fi
    info "verifying airgap tarball"
    AIRGAP_CHECKSUM_ACTUAL=$(sha256sum "${TMP_AIRGAP_TARBALL}" | awk '{print $1}')
    if [ "${AIRGAP_CHECKSUM_EXPECTED}" != "${AIRGAP_CHECKSUM_ACTUAL}" ]; then
        fatal "download sha256 does not match ${AIRGAP_CHECKSUM_EXPECTED}, got ${AIRGAP_CHECKSUM_ACTUAL}"
    fi
}

# install_airgap_tarball moves the airgap image tarball into place.
install_airgap_tarball() {
    if [ -z "${AIRGAP_CHECKSUM_EXPECTED}" ]; then
        return
    fi
    mkdir -p "${INSTALL_RKE2_AGENT_IMAGES_DIR}"
    # releases that provide zst artifacts can read from the compressed archive; older releases
    # that produce only gzip artifacts need to have the tarball decompressed ahead of time
    if grep -sqF '.tar.zst' "${TMP_AIRGAP_CHECKSUMS}" || [ "${AIRGAP_TARBALL_FORMAT}" = "zst" ]; then
        info "installing airgap tarball to ${INSTALL_RKE2_AGENT_IMAGES_DIR}"
        mv -f "${TMP_AIRGAP_TARBALL}" "${INSTALL_RKE2_AGENT_IMAGES_DIR}/rke2-images.${SUFFIX}.tar.zst"
    else
        info "decompressing airgap tarball to ${INSTALL_RKE2_AGENT_IMAGES_DIR}"
        gzip -dc "${TMP_AIRGAP_TARBALL}" > "${INSTALL_RKE2_AGENT_IMAGES_DIR}/rke2-images.${SUFFIX}.tar"
    fi
    # Search for and install additional rke2 images
    for IMAGE in "${INSTALL_RKE2_ARTIFACT_PATH}"/rke2-images-*."${SUFFIX}"*; do 
        if [ -f "${IMAGE}" ]; then
            info "Installing airgap image from ${IMAGE}"
            cp "${IMAGE}" "${INSTALL_RKE2_AGENT_IMAGES_DIR}"
        fi
    done
}

# install_dev_rpm orchestrates the installation of RKE2 unsigned development rpms
install_dev_rpm() {
    rpm_installer="${1:-yum}"
    distro="${2:-centos7}"

    [ -z "${TMP_DIR}" ] && setup_tmp

    case "${rpm_installer}" in
        *zypper*)
            ${rpm_installer} install --allow-unsigned-rpm -y \
                "${STORAGE_URL}/rke2-common-${INSTALL_RKE2_COMMIT}.${distro}.rpm" \
                "${STORAGE_URL}/rke2-${INSTALL_RKE2_TYPE}-${INSTALL_RKE2_COMMIT}.${distro}.rpm"
            ;;
        *)
            download_dev_rpm "${TMP_DIR}/rke2-common-${INSTALL_RKE2_COMMIT}.${distro}.rpm"
            download_dev_rpm "${TMP_DIR}/rke2-${INSTALL_RKE2_TYPE}-${INSTALL_RKE2_COMMIT}.${distro}.rpm"
            ${rpm_installer} install -y  "${TMP_DIR}"/*.rpm
            ;;
    esac

    if [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst" ]; then
        return
    fi

    download_airgap_checksums
    download_airgap_tarball
    verify_airgap_tarball
    install_airgap_tarball
}

# do_install_rpm builds a yum repo config from the channel and version to be installed,
# and calls yum to install the required packages.
do_install_rpm() {
    . /etc/os-release
    if [ -r /etc/redhat-release ] || [ -r /etc/centos-release ] || [ -r /etc/oracle-release ] || [ -r /etc/system-release ] || [ "${ID_LIKE%%[ ]*}" = "suse"  ]; then
        repodir=/etc/yum.repos.d
        if [ -d /etc/zypp/repos.d ]; then
            repodir=/etc/zypp/repos.d
        fi
        if [ "${ID_LIKE%%[ ]*}" = "suse" ]; then
            # create the /var/lib/rpm-state in SLE systems to fix the prein selinux macro
            if [ "${TRANSACTIONAL_UPDATE=false}" != "true" ] && [ -x /usr/sbin/transactional-update ]; then
                transactional_update_run="transactional-update --no-selfupdate -d run"
            fi
            ${transactional_update_run} mkdir -p /var/lib/rpm-state
            # configure infix and rpm_installer
            rpm_site_infix=microos
            if [ "${VARIANT_ID:-}" = sle-micro ] || [ "${ID:-}" = sle-micro ] || [ "${ID:-}" = sl-micro ]; then
                rpm_site_infix=slemicro
                package_installer=zypper
            fi
            rpm_installer="zypper --gpg-auto-import-keys"
            if [ "${TRANSACTIONAL_UPDATE=false}" != "true" ] && [ -x /usr/sbin/transactional-update ]; then
                rpm_installer="transactional-update --no-selfupdate -d run ${rpm_installer}"
            fi
        else
            maj_ver=$(echo "$VERSION_ID" | sed -E -e "s/^([0-9]+)\.?[0-9]*$/\1/")
            case ${maj_ver} in
                7|8|9|10)
                    :
                    ;;
                2023) # detect amazon linux 2023 distro
                    maj_ver="8"
                    ;;
                *) # set default distro to centos 8, for edge cases such as fedora
                    maj_ver="8"
                    ;;
            esac
            rpm_site_infix=centos/${maj_ver}
            rpm_installer="yum"
        fi

    fi
    case "${INSTALL_RKE2_CHANNEL}" in
        v*.*)
            # We are operating with a version-based channel, so we should parse our version out
            rke2_majmin=$(echo "${INSTALL_RKE2_CHANNEL}" | sed -E -e "s/^v([0-9]+\.[0-9]+).*/\1/")
            rke2_rpm_channel=$(echo "${INSTALL_RKE2_CHANNEL}" | sed -E -e "s/^v[0-9]+\.[0-9]+-(.*)/\1/")
            # If our regex fails to capture a "sane" channel out of the specified channel, fall back to `stable`
            if [ "${rke2_rpm_channel}" = ${INSTALL_RKE2_CHANNEL} ]; then
                info "using stable RPM repositories"
                rke2_rpm_channel="stable"
            fi
            ;;
        *)
            get_release_version
            rke2_majmin=$(echo "${INSTALL_RKE2_VERSION}" | sed -E -e "s/^v([0-9]+\.[0-9]+).*/\1/")
            rke2_rpm_channel=${1}
            ;;
    esac
    info "using ${rke2_majmin} series from channel ${rke2_rpm_channel}"
    rpm_site="rpm.rancher.io"
    if [ "${rke2_rpm_channel}" = "testing" ]; then
        rpm_site="rpm-${rke2_rpm_channel}.rancher.io"
    fi

    rm -f ${repodir}/rancher-rke2*.repo
    cat <<-EOF >"${repodir}/rancher-rke2.repo"
[rancher-rke2-common-${rke2_rpm_channel}]
name=Rancher RKE2 Common (${1})
baseurl=https://${rpm_site}/rke2/${rke2_rpm_channel}/common/${rpm_site_infix}/noarch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://${rpm_site}/public.key
EOF

    if [ -z "${INSTALL_RKE2_COMMIT}" ]; then
    cat <<-EOF >>"${repodir}/rancher-rke2.repo"
[rancher-rke2-${rke2_majmin}-${rke2_rpm_channel}]
name=Rancher RKE2 ${rke2_majmin} (${1})
baseurl=https://${rpm_site}/rke2/${rke2_rpm_channel}/${rke2_majmin}/${rpm_site_infix}/$(uname -m)
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://${rpm_site}/public.key
EOF
    fi

    if rpm -q --quiet rke2-selinux; then
            # remove rke2-selinux module in el9 before upgrade to allow container-selinux to upgrade safely
            if check_available_upgrades container-selinux && check_available_upgrades rke2-selinux && check_breaking_version container-selinux 2 189; then
                MODULE_PRIORITY=$(semodule --list=full | grep rke2 | cut -f1 -d" ")
                if [ -n "${MODULE_PRIORITY}" ]; then
                    semodule -X $MODULE_PRIORITY -r rke2 || true
                fi
            fi
    fi

    if [ -z "${INSTALL_RKE2_VERSION}" ] && [ -z "${INSTALL_RKE2_COMMIT}" ]; then
        ${rpm_installer} install -y "rke2-${INSTALL_RKE2_TYPE}"
    elif [ -n "${INSTALL_RKE2_COMMIT}" ]; then
        rel_distro="$(echo "${rpm_site_infix}" | tr -d /)"
        install_dev_rpm "${rpm_installer}" "${rel_distro}"
    else
        rke2_rpm_version=$(echo "${INSTALL_RKE2_VERSION}" | sed -E -e "s/[\+-]/~/g" | sed -E -e "s/v(.*)/\1/")
        if [ -n "${INSTALL_RKE2_RPM_RELEASE_VERSION}" ]; then
            ${rpm_installer} install -y "rke2-${INSTALL_RKE2_TYPE}-${rke2_rpm_version}-${INSTALL_RKE2_RPM_RELEASE_VERSION}"
        else
            ${rpm_installer} install -y "rke2-${INSTALL_RKE2_TYPE}-${rke2_rpm_version}"
        fi
    fi
}

check_breaking_version() {
  maj=$2
  min=$3

  current_maj=$(rpm -qi $1 | awk -F': ' '/Version/ {print $2}' | sed -E -e "s/^([0-9]+)\.([0-9]+).*/\1/")
  current_min=$(rpm -qi $1 | awk -F': ' '/Version/ {print $2}' | sed -E -e "s/^([0-9]+)\.([0-9]+).*/\2/")

  if [ "${current_maj}" == "${maj}" ] && [ $current_min -le $min ]; then
    return 0
  fi

  return 1
}

check_available_upgrades() {
    . /etc/os-release
    set +e
    if [ "${ID_LIKE%%[ ]*}" = "suse" ]; then
        available_upgrades=$(zypper -q -t -s 11 se -s -u --type package $1 | tail -n 1 | grep -v "No matching" | awk '{print $3}')
    else
        available_upgrades=$(yum -q --refresh list $1 --upgrades | tail -n 1 | awk '{print $2}')
    fi
    set -e
    if [ -n "${available_upgrades}" ]; then
        return 0
    fi
    return 1
}

do_install_tar() {
    setup_tmp

    if [ -n "${INSTALL_RKE2_ARTIFACT_PATH}" ]; then
        stage_local_checksums
        stage_local_airgap_tarball
        stage_local_tarball
    else
        get_release_version
        info "using ${INSTALL_RKE2_VERSION:-commit $INSTALL_RKE2_COMMIT} as release"
        download_airgap_checksums
        download_airgap_tarball
        download_checksums
        download_tarball
    fi

    verify_airgap_tarball
    install_airgap_tarball
    verify_tarball
    unpack_tarball

    if [ -z "${INSTALL_RKE2_SKIP_RELOAD}" ]; then
        systemctl daemon-reload
    fi
}

setup_fapolicy_rules() {
    if [ -r /etc/redhat-release ] || [ -r /etc/centos-release ] || [ -r /etc/oracle-release ] || [ -r /etc/rocky-release ] || [ -r /etc/system-release ]; then
        verify_fapolicyd || return 0
        # setting rke2 fapolicyd rules
        cat <<-EOF >"/etc/fapolicyd/rules.d/80-rke2.rules"
allow perm=any all : dir=/var/lib/rancher/
allow perm=any all : dir=/opt/cni/
allow perm=any all : dir=/run/k3s/
allow perm=any all : dir=/var/lib/kubelet/
EOF
        if [ -z "${INSTALL_RKE2_SKIP_RELOAD}" ]; then
            fagenrules --load || fatal "failed to load rke2 fapolicyd rules"
            systemctl restart fapolicyd
        fi
    fi
}

do_install() {
    setup_env
    check_method_conflict
    setup_arch
    if [ -z "${INSTALL_RKE2_ARTIFACT_PATH}" ]; then
        verify_downloader curl || verify_downloader wget || fatal "can not find curl or wget for downloading files"
    fi

    case ${INSTALL_RKE2_METHOD} in
    yum | rpm | dnf)
        do_install_rpm "${INSTALL_RKE2_CHANNEL}"
        ;;
    *)
        do_install_tar "${INSTALL_RKE2_CHANNEL}"
        ;;
    esac
    if [ -z "${INSTALL_RKE2_SKIP_FAPOLICY}" ]; then
        setup_fapolicy_rules
    fi
}

do_install
exit 0

 

 

  • Configuring the Linux Installation Script : 설치에 적용할 수 있는 환경변수 - Docs
    • INSTALL_RKE2_VERSION : GitHub에서 다운로드할 RKE2 버전 - stable(기본값),
    • INSTALL_RKE2_TYPE : 생성할 systemd 서비스의 유형 - server(기본값), agent
    • INSTALL_RKE2_CHANNEL_URL : RKE2 다운로드 URL을 가져오기 위한 채널 URL - https://update.rke2.io/v1-release/channels (기본값)
    • INSTALL_RKE2_CHANNEL : RKE2 다운로드 URL을 가져오는 데 사용할 채널 - stable(기본값), latest, testing
    • INSTALL_RKE2_METHOD : 사용할 설치 방법 - rpm(RPM 기반 시스템 경우 기본값), tar(그외 경우 기본값)

 

# systemd 기반 시스템에 서비스로 편리하게 설치할 수 있는 설치 스크립트를 제공 : 서비스와 바이너리 파일이 컴퓨터에 설치
# https://docs.rke2.io/install/methods
curl -sfL https://get.rke2.io --output install.sh
chmod +x install.sh
INSTALL_RKE2_CHANNEL=v1.33 ./install.sh
...
Installed:
  rke2-common-1.33.7~rke2r3-0.el9.aarch64             rke2-selinux-0.22-1.el9.noarch             rke2-server-1.33.7~rke2r3-0.el9.aa

# rke2 버전 확인
rke2 --version
rke2 version v1.33.7+rke2r3 (7e4fd1a82edf497cab91c220144619bbad659cf4)
go version go1.24.11 X:boringcrypto

# repo 추가 확인
dnf repolist
rancher-rke2-1.33-stable                                                  Rancher RKE2 1.33 (v1.33)
rancher-rke2-common-stable                                                Rancher RKE2 Common (v1.33)

tree /etc/yum.repos.d/
cat /etc/yum.repos.d/rancher-rke2.repo
[rancher-rke2-common-stable]
name=Rancher RKE2 Common (v1.33)
baseurl=https://rpm.rancher.io/rke2/stable/common/centos/9/noarch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://rpm.rancher.io/public.key
[rancher-rke2-1.33-stable]
name=Rancher RKE2 1.33 (v1.33)
baseurl=https://rpm.rancher.io/rke2/stable/1.33/centos/9/aarch64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://rpm.rancher.io/public.key

# 디렉터리 생성 확인
tree /etc/rancher/
tree /var/lib/rancher/

# rke2 명령 확인
# https://docs.rke2.io/install/configuration#running-the-binary-directly
# rke2 server : Run the RKE2 management server, which will also launch the Kubernetes control plane components such as the API server, controller-manager, and scheduler. Only Supported on Linux.
# rke2 agent : Run the RKE2 node agent. This will cause RKE2 to run as a worker node, launching the Kubernetes node services kubelet and kube-proxy. Supported on Linux and Windows.
rke2 --h
   server           Run management server
   agent            Run node agent


# RKE2 설정 : cni 플러그인(canal) 등
# https://docs.rke2.io/install/configuration
# https://docs.rke2.io/advanced
cat << EOF > /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"

debug: true

cni: canal

bind-address: 192.168.10.11
advertise-address: 192.168.10.11
node-ip: 192.168.10.11

disable-cloud-controller: true

disable:
  - servicelb
  - rke2-coredns-autoscaler
  - rke2-ingress-nginx
  - rke2-snapshot-controller
  - rke2-snapshot-controller-crd
  - rke2-snapshot-validation-webhook
EOF
cat /etc/rancher/rke2/config.yaml

# canal cni 플러그인 helm chart values 파일 작성
# https://docs.rke2.io/networking/basic_network_options
# https://github.com/rancher/rke2-charts/blob/main-source/packages/rke2-canal/charts/values.yaml
mkdir -p /var/lib/rancher/rke2/server/manifests/
cat << EOF > /var/lib/rancher/rke2/server/manifests/rke2-canal-config.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-canal
  namespace: kube-system
spec:
  valuesContent: |-
    flannel:
      iface: "enp0s9"
EOF

# coredns 의 autoscaler 미설치를 위한 helm chart values 파일 작성
# https://docs.rke2.io/add-ons/helm#customizing-packaged-components-with-helmchartconfig
# https://github.com/rancher/rke2-charts/tree/main/charts/rke2-coredns/rke2-coredns/1.45.200
cat << EOF > /var/lib/rancher/rke2/server/manifests/rke2-coredns-config.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-coredns
  namespace: kube-system
spec:
  valuesContent: |-
    autoscaler:
      enabled: false
EOF
    

# 모니터링 : 신규 터미널창
watch -d pstree -a
journalctl -u rke2-server -f


# RKE2 시작 : 2분 정도 소요 -> coredns 파드까지 정상화 대략 1~2분 추가 소요
systemctl enable --now rke2-server.service
systemctl status rke2-server --no-pager


# 프로세스 확인
pstree -a | grep -v color | grep 'rke2$' -A5
pstree -a | grep -v color | grep 'containerd-shim ' -A2


# 자격증명 파일 복사
mkdir ~/.kube
ls -l /etc/rancher/rke2/rke2.yaml
cp /etc/rancher/rke2/rke2.yaml ~/.kube/config

# /etc/rancher 디렉터리 확인
tree /etc/rancher/
cat /etc/rancher/node/password
cat /etc/rancher/rke2/config.yaml
cat /etc/rancher/rke2/rke2-pss.yaml 

# 바이너리 파일 확인
tree /var/lib/rancher/rke2/bin/
├── containerd
├── containerd-shim-runc-v2
├── crictl
├── ctr
├── kubectl
├── kubelet
└── runc

# PATH 안 건드리고 표준 위치로 바이너리 노출 설정 : 심볼릭 링크 방식 
ln -s /var/lib/rancher/rke2/bin/containerd /usr/local/bin/containerd
ln -s /var/lib/rancher/rke2/bin/kubectl /usr/local/bin/kubectl
ln -s /var/lib/rancher/rke2/bin/crictl /usr/local/bin/crictl
ln -s /var/lib/rancher/rke2/bin/runc /usr/local/bin/runc
ln -s /var/lib/rancher/rke2/bin/ctr /usr/local/bin/ctr
ln -s /var/lib/rancher/rke2/agent/etc/crictl.yaml /etc/crictl.yaml

runc --version
containerd --version
kubectl version

# 편의성 설정
source <(kubectl completion bash)
alias k=kubectl
complete -F __start_kubectl k
echo 'source <(kubectl completion bash)' >> /etc/profile
echo 'alias k=kubectl' >> /etc/profile
echo 'complete -F __start_kubectl k' >> /etc/profile

k9s

# 확인
kubectl cluster-info -v=6
Kubernetes control plane is running at https://192.168.10.11:6443

# 노드, 파드 정보 확인
kubectl get node -owide
NAME        STATUS   ROLES                AGE   VERSION          INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                      KERNEL-VERSION                  CONTAINER-RUNTIME
k8s-node1   Ready    control-plane,etcd   15m   v1.34.3+rke2r3   192.168.10.11   <none>        Rocky Linux 9.6 (Blue Onyx)   5.14.0-570.52.1.el9_6.aarch64   containerd://2.1.5-k3s1

helm list -A
NAME               	NAMESPACE  	REVISION	UPDATED                                	STATUS  	CHART                             	APP VERSION
rke2-canal         	kube-system	1       	2026-02-09 14:30:45.002896818 +0000 UTC	deployed	rke2-canal-v3.31.3-build2026011900	v3.31.3
rke2-coredns       	kube-system	1       	2026-02-09 14:30:44.989504642 +0000 UTC	deployed	rke2-coredns-1.45.008             	1.13.1
rke2-metrics-server	kube-system	1       	2026-02-09 14:31:13.07011618 +0000 UTC 	deployed	rke2-metrics-server-3.13.006      	0.8.0
rke2-runtimeclasses	kube-system	1       	2026-02-09 14:31:17.059719859 +0000 UTC	deployed	rke2-runtimeclasses-0.1.000       	0.1.0

kubectl get pod -A
NAMESPACE     NAME                                         READY   STATUS      RESTARTS   AGE
kube-system   etcd-k8s-node1                               1/1     Running     0          2m31s
kube-system   helm-install-rke2-canal-8ck96                0/1     Completed   0          2m42s
kube-system   helm-install-rke2-coredns-mhw4x              0/1     Completed   0          2m42s
kube-system   helm-install-rke2-metrics-server-rhqcp       0/1     Completed   0          2m42s
kube-system   helm-install-rke2-runtimeclasses-7m69w       0/1     Completed   0          2m42s
kube-system   kube-apiserver-k8s-node1                     1/1     Running     0          2m31s
kube-system   kube-controller-manager-k8s-node1            1/1     Running     0          2m31s
kube-system   kube-proxy-k8s-node1                         1/1     Running     0          2m31s
kube-system   kube-scheduler-k8s-node1                     1/1     Running     0          2m31s
kube-system   rke2-canal-dkw2n                             2/2     Running     0          2m22s
kube-system   rke2-coredns-rke2-coredns-784bcb7f4d-tpt2d   1/1     Running     0          2m22s
kube-system   rke2-metrics-server-7b59bd8854-m5w2c         1/1     Running     0          114s

 

 

/var/lib/rancher/rke2 디렉터리 : addon helm chart, 인증서

# 디렉터리 확인
tree /var/lib/rancher/rke2 -L 1
├── agent
├── bin -> /var/lib/rancher/rke2/data/v1.34.3-rke2r3-5b8349de68df/bin
├── data
└── server

# server 디렉터리
tree /var/lib/rancher/rke2/server/
tree /var/lib/rancher/rke2/server/ -L 1
├── agent-token -> /var/lib/rancher/rke2/server/token
├── cred           # kubeconfig 정보들
├── db             # etcd snapshots, etcd member snap/wal
├── etc            
├── manifests      # helm chart manifests by helm controller
├── node-token -> /var/lib/rancher/rke2/server/token
├── tls            # 인증서 관련 파일들
└── token

## A token that can be used to register other server or agent nodes will be created at /var/lib/rancher/rke2/server/node-token
ls -l /var/lib/rancher/rke2/server/
cat /var/lib/rancher/rke2/server/node-token
cat /var/lib/rancher/rke2/server/token
K1087d08f6b3f21e75c6835e1b959d634dfbe44970c7b1c93c61d8c510f8076822f::server:466038852cf7cdcd389872ba4ab64c04

## helm chart manifests + values 포함
cat /var/lib/rancher/rke2/server/manifests/rke2-coredns.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChart
...
  set:
    global.clusterCIDR: 10.42.0.0/16
    global.clusterCIDRv4: 10.42.0.0/16

# addon, helm controller 관련 crd 확인
kubectl get crd | grep -E 'helm|addon'
addons.k3s.cattle.io                                    2026-02-14T07:33:46Z
helmchartconfigs.helm.cattle.io                         2026-02-14T07:33:46Z
helmcharts.helm.cattle.io                               2026-02-14T07:33:46Z

kubectl get helmcharts.helm.cattle.io -n kube-system -owide
NAME                  REPO   CHART   VERSION   TARGETNAMESPACE   BOOTSTRAP   FAILED   JOB
rke2-canal                                                       true        False    helm-install-rke2-canal
rke2-coredns                                                     true        False    helm-install-rke2-coredns
rke2-metrics-server                                                          False    helm-install-rke2-metrics-server
rke2-runtimeclasses                                                          False    helm-install-rke2-runtimeclasses

kubectl get job -n kube-system
NAME                               STATUS     COMPLETIONS   DURATION   AGE
helm-install-rke2-canal            Complete   1/1           21s        118m
helm-install-rke2-coredns          Complete   1/1           20s        118m
helm-install-rke2-metrics-server   Complete   1/1           52s        118m
helm-install-rke2-runtimeclasses   Complete   1/1           51s        118m

kubectl get helmchartconfigs -n kube-system
NAME           AGE
rke2-canal     119m
rke2-coredns   119m

kubectl describe helmchartconfigs -n kube-system rke2-canal
...
Spec:
  Failure Policy:  reinstall
  Values Content:  flannel:
  iface: "enp0s9"

kubectl get addons.k3s.cattle.io -n kube-system
NAME                  SOURCE                                                            CHECKSUM
rke2-canal            /var/lib/rancher/rke2/server/manifests/rke2-canal.yaml            2de126b0695f5c424eba2b8e7c45c67a757edac6ecccce7a6141228e6c2fd852
rke2-canal-config     /var/lib/rancher/rke2/server/manifests/rke2-canal-config.yaml     122823431e18d9f23621508c24bff9aff307f46b4d6d9208bbbbf23311db058e
rke2-coredns          /var/lib/rancher/rke2/server/manifests/rke2-coredns.yaml          523c74e141f7722f3721ce5b3b6556241199bf3aed9b781a5cbfeebd2fe595f4
rke2-coredns-config   /var/lib/rancher/rke2/server/manifests/rke2-coredns-config.yaml   9c8e2bbb7603c69c233c9343c516d3d302fd59b758e2e7084c7ab08b5bfba0e4
rke2-metrics-server   /var/lib/rancher/rke2/server/manifests/rke2-metrics-server.yaml   4fe80fe31c6cc3d5f1cb7cd19f4d3181f1c810751ed995fa1ca5d9a560c436f8
rke2-runtimeclasses   /var/lib/rancher/rke2/server/manifests/rke2-runtimeclasses.yaml   7915375038cf0683ea2ee558d01adb9a478d1c51ec3f0a508b9082cd4f0a7145


## 인증서 관련 파일들
cat /var/lib/rancher/rke2/server/tls/server-ca.crt | openssl x509 -text -noout # k8s ca 인증서
cat /var/lib/rancher/rke2/server/tls/serving-kube-apiserver.crt | openssl x509 -text -noout # apiserver web server 인증서
        Issuer: CN=rke2-server-ca@1770533252
        Validity
            Not Before: Feb  8 06:47:32 2026 GMT
            Not After : Feb  8 06:47:32 2027 GMT
        Subject: CN=kube-apiserver
        ...
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Subject Alternative Name: 
                DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, DNS:localhost, DNS:k8s-node1, IP Address:127.0.0.1, IP Address:0:0:0:0:0:0:0:1, IP Address:192.168.10.11, IP Address:192.168.10.11, IP Address:10.43.0.1


# data 디렉터리
tree /var/lib/rancher/rke2/data/
tree /var/lib/rancher/rke2/data/ -L 2
└── v1.34.3-rke2r3-5b8349de68df
    ├── bin        # 핵심 구성요소 관련 바이너리 파일들
    └── charts     # helm charts by helm controller

## helm chart manifests
cat /var/lib/rancher/rke2/data/v1.34.3-rke2r3-5b8349de68df/charts/rke2-coredns.yaml


# agent 디렉터리
tree /var/lib/rancher/rke2/agent/ | more
tree /var/lib/rancher/rke2/agent/ -L 3
├── client-ca.crt                             # client 관련 kubeconfig 및 인증서 파일들
...
├── containerd                                # containerd root 디렉터리 , (설정값) root = "/var/lib/rancher/rke2/agent/containerd"
│   ├── bin
│   ├── containerd.log
│   ├── io.containerd.content.v1.content
│   │   ├── blobs
│   │   └── ingest
    ...
│   └── tmpmounts
├── etc                                       # 주요 설정 파일
│   ├── containerd
│   │   └── config.toml
│   ├── crictl.yaml
│   └── kubelet.conf.d
│       └── 00-rke2-defaults.conf
├── images
│   ├── cloud-controller-manager-image.txt
│   ├── etcd-image.txt
│   ├── kube-apiserver-image.txt
│   ├── kube-controller-manager-image.txt
│   ├── kube-proxy-image.txt
│   ├── kube-scheduler-image.txt
│   └── runtime-image.txt
├── kubelet.kubeconfig                        # client 관련 kubeconfig 및 인증서 파일들
...
├── logs                                      # kubelet 로그
│   └── kubelet.log
├── pod-manifests                             # static pod manifests
│   ├── cloud-controller-manager.yaml
│   ├── etcd.yaml
│   ├── kube-apiserver.yaml
│   ├── kube-controller-manager.yaml
│   ├── kube-proxy.yaml
│   └── kube-scheduler.yaml
...

## crictl.yaml 파일 확인 : k3s 포함된 경로 확인!
crictl ps
cat /var/lib/rancher/rke2/agent/etc/crictl.yaml
runtime-endpoint: unix:///run/k3s/containerd/containerd.sock

ln -s /var/lib/rancher/rke2/agent/etc/crictl.yaml /etc/crictl.yaml
crictl ps
crictl images

## rke2 에서 config.toml 은 직접 수정 비권장 -> 직접 수정하면 rke2/k3s 재시작 시 덮어써짐!
cat /var/lib/rancher/rke2/agent/etc/containerd/config.toml
# File generated by rke2. DO NOT EDIT. Use config.toml.tmpl instead.
version = 3
root = "/var/lib/rancher/rke2/agent/containerd"
...
[plugins.'io.containerd.cri.v1.images'.registry]
  config_path = "/var/lib/rancher/rke2/agent/etc/containerd/certs.d"

## containerd registry 설정 디렉터리/파일이 현재는 없음
ls -l /var/lib/rancher/rke2/agent/etc/containerd/certs.d
ls: cannot access '/var/lib/rancher/rke2/agent/etc/containerd/certs.d': No such file or directory

## 컨테이너 이미지 명
grep -H '' /var/lib/rancher/rke2/agent/images/*
/var/lib/rancher/rke2/agent/images/etcd-image.txt:index.docker.io/rancher/hardened-etcd:v3.6.7-k3s1-build20260126
/var/lib/rancher/rke2/agent/images/kube-apiserver-image.txt:index.docker.io/rancher/hardened-kubernetes:v1.34.3-rke2r3-build20260127
/var/lib/rancher/rke2/agent/images/kube-controller-manager-image.txt:index.docker.io/rancher/hardened-kubernetes:v1.34.3-rke2r3-build20260127
/var/lib/rancher/rke2/agent/images/kube-proxy-image.txt:index.docker.io/rancher/hardened-kubernetes:v1.34.3-rke2r3-build20260127
/var/lib/rancher/rke2/agent/images/kube-scheduler-image.txt:index.docker.io/rancher/hardened-kubernetes:v1.34.3-rke2r3-build20260127
/var/lib/rancher/rke2/agent/images/runtime-image.txt:index.docker.io/rancher/rke2-runtime:v1.34.3-rke2r3

crictl images

## kubelet config : 보안 권장 설정 및 엔터프라이즈 권장 설정 반영되어 있음
cat /var/lib/rancher/rke2/agent/etc/kubelet.conf.d/00-rke2-defaults.conf
address: 192.168.10.11
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: /var/lib/rancher/rke2/agent/client-ca.crt
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
cgroupDriver: systemd
clusterDNS:
- 10.43.0.10
clusterDomain: cluster.local
containerRuntimeEndpoint: unix:///run/k3s/containerd/containerd.sock
cpuManagerReconcilePeriod: 10s
crashLoopBackOff: {}
evictionHard:
  imagefs.available: 5%
  nodefs.available: 5%
evictionMinimumReclaim:
  imagefs.available: 10%
  nodefs.available: 10%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: false                  # 원래 쿠버네티스는 스왑(Swap) 메모리가 켜져 있으면 실행되지 않지만, RKE2는 이를 허용하도록 설정되어 있습니다.
fileCheckFrequency: 20s
healthzBindAddress: 127.0.0.1
httpCheckFrequency: 20s
imageMaximumGCAge: 0s
imageMinimumGCAge: 2m0s
kind: KubeletConfiguration
logging:
  flushFrequency: 5s
  format: text
  options:
    json:
      infoBufferSize: "0"
    text:
      infoBufferSize: "0"
  verbosity: 0
memorySwap: {}
nodeStatusReportFrequency: 5m0s
nodeStatusUpdateFrequency: 10s     # 10초마다 "나 살아있어요"라고 컨트롤플레인 노드에 보고
resolvConf: /etc/resolv.conf
runtimeRequestTimeout: 2m0s
serializeImagePulls: false         # 이미지 병렬 pull 허용
shutdownGracePeriod: 0s
shutdownGracePeriodCriticalPods: 0s
staticPodPath: /var/lib/rancher/rke2/agent/pod-manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
tlsCertFile: /var/lib/rancher/rke2/agent/serving-kubelet.crt
tlsPrivateKeyFile: /var/lib/rancher/rke2/agent/serving-kubelet.key
volumeStatsAggPeriod: 1m0s

## kubelet 로그
tail -f /var/lib/rancher/rke2/agent/logs/kubelet.log

'Kubernetes' 카테고리의 다른 글

[6주차] Kubespray offline 설치  (0) 2026.02.15
[5주차] Kubespary HA & Upgrade  (0) 2026.02.06
[4주차] Kubespary 배포 분석  (0) 2026.02.06
[3주차] Kubeadm & K8S Upgrade  (0) 2026.01.20
[2주차] Ansible 기초  (0) 2026.01.12
공지사항
최근에 올라온 글
최근에 달린 댓글
Total
Today
Yesterday
링크
TAG more
«   2026/06   »
일 월 화 수 목 금 토
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30
글 보관함