kubeadm集群重置

通常 kubeadm 通过 kubeadm init 或者 kubeadm join 来实现集群初始化和节点添加。但是,我在 基于DNS轮询构建高可用Kubernetes 部署时没有与时俱进正确配置适配 Kubernetes v1.24 的 containerd运行时(runtime) 网络,导致容器无法启动。后续虽然做了很多手工修复,但是感觉离最终解决还是差了一点点。所以,改为采用重置集群,重新开始部署方式。

kubeadm reset 命令提供了对本地主机重置清理能力,是 kubeadm initkubeadm join 逆过程:

  • --dry-run 参数可以模拟运行,显示出 reset 指令会做哪些操作,这样方便预估影响。建议在实际运行 kubeadm reset 之前先使用这个参数模拟一下

  • kubeadm reset 会清理掉节点本地文件系统上之前通过 kubeadm initkubeadm join 命令创建的文件。例如,对于管控节点 reset 也会移除本地运行的 etcd 成员(如果是本地堆叠的etcd),但是如果采用外部etcd,则不会清理外部etcd集群中任何数据,此时需要独立etcd清理(见下文)

清理管控平面节点:

清理kubernetes节点
kubeadm reset

提示信息:

清理kubernetes节点
[reset] Reading configuration from the cluster...
[reset] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
W0712 10:51:25.713207  125389 preflight.go:55] [reset] WARNING: Changes made to this host by 'kubeadm init' or 'kubeadm join' will be reverted.
[reset] Are you sure you want to proceed? [y/N]: y
[preflight] Running pre-flight checks
W0712 10:51:28.605539  125389 removeetcdmember.go:84] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
[reset] Deleting contents of directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]

The reset process does not clean CNI configuration. To do so, you must remove /etc/cni/net.d

The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually by using the "iptables" command.

If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables.

The reset process does not clean your kubeconfig files and you must remove them manually.
Please, check the contents of the $HOME/.kube/config file.

对于 私有云部署TLS认证的etcd集群 ,etcd是独立运行的外部etcd集群,需要独立命令清理:

清理kubernetes外部etcd集群数据
etcdctl del "" --prefix

参考