nerdctl¶
nerdctl 是 containerd 子项目，提供Docker兼容的CLI命令:
类似 Docker Atlas 的交互方式
支持Docker Compose
支持rootless模式
支持 lazy-pulling (例如 用于延迟拉取(Lazy Pulling)的eStargz(容器镜像层标准兼容扩展) )
支持加密镜像
支持P2P镜像分发 ( IPFS )
支持容器镜像签名和验证
安装¶
从 nerdctl releases 下载软件包:
ARCH=amd64
VER=0.22.2
#命令行下载因转跳失败，所以还是浏览器下载后解压缩
#curl https://github.com/containerd/nerdctl/releases/download/v${VER}/nerdctl-${VER}-linux-${ARCH}.tar.gz -o nerdctl-${VER}-linux-${ARCH}.tar.gz
tar xfz nerdctl-${VER}-linux-${ARCH}.tar.gz
sudo mv nerdctl /usr/bin/
官方下载有两种软件包:
Minimal (nerdctl-0.22.0-linux-amd64.tar.gz): nerdctl only
Full (nerdctl-full-0.22.0-linux-amd64.tar.gz): Includes dependencies such as containerd, runc, and CNI
我安装的是Minimal版本，其他组件按需各自安装
备注
在 macOS 上运行 Colima 可以获得开源的容器运行环境，支持采用 containerd运行时(runtime) 作为运行时。
快速起步¶
有root权限方式:
sudo systemctl enable --now containerd sudo nerdctl run -d --name nginx -p 80:80 nginx:alpine
备注
由于我是安装了 containerd运行时(runtime) 服务，是通过root模式运行的，所以后续操作以这个root权限方式为准。也就是后续 nerdctl 命令操作都是使用 sudo nerdctl
这里我遇到一个报错，显示GFW阻挡了镜像下载:
FATA[0016] failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/b6/b692a91e4e1582db97076184dae0b2f4a7a86b68c4fe6f91affa50ae06369bf5/data?verify=1660148606-x7zt9HGf2yYAnJhu906wV%2FXXVwA%3D": net/http: TLS handshake timeout
通过配置 配置Docker使用代理 环境变量:
export HTTP_PROXY=http://192.168.6.200:3128/
export HTTPS_PROXY=http://192.168.6.200:3128/
然后再次执行就能正常下载镜像并运行容器。此时检查:
sudo nerdctl ps
备注
可以看到本地运行了容器(类似docker):
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
71696cbb5444 docker.io/library/nginx:latest "/docker-entrypoint.…" About a minute ago Up 0.0.0.0:80->80/tcp nginx
备注
直接使用 nerdctl run 是在本地运行容器，没有运行在Kuberntes集群，所以不需要使用 -n k8s.io
无root权限方式(端口1024以上):
containerd-rootless-setuptool.sh install nerdctl run -d --name nginx -p 8080:80 nginx:alpine
容器探索¶
我在下文尝试通过 Buildkit 构建镜像遇到容器内部不能通网络的问题，所以，我返回来先验证官方镜像是否能够正常运行，并排查网络
进入
nginx容器:sudo nerdctl exec -it nginx /bin/bash
验证可以进入容器内部，不过此时容器内部缺少很多网络排查工具，甚至都没有提供ps命令
检查发现，原来默认 nginx:lastest 镜像是基于 Ubuntu Linux ，可以使用 APT包管理 ，并且能够通internet。这就好办了，安装工具:
apt update
apt upgrade
apt install iproute2
检查:
ip link
显示:
...
2: eth0@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
link/ether 16:4f:d5:8c:e0:86 brd ff:ff:ff:ff:ff:ff link-netnsid 0
路由:
ip route
显示:
default via 10.4.0.1 dev eth0
10.4.0.0/24 dev eth0 proto kernel scope link src 10.4.0.2
为什么容器内部默认就能够连接外网呢？这明显就是类似 Docker Atlas 的NAT网络么
在物理主机上检查:
brctl show
果然可以看到 nerdctl 启动了一个交换网络:
bridge name bridge id STP enabled interfaces
nerdctl0 8000.d20bcb07c7c8 no veth482da55d
实践:镜像¶
nerdctl 对镜像的制作是采用 Buildkit 实现的，请先安装 buildkit 并运行 buildkitd 服务进程
使用root身份运行buildkitd，启动后工作在前台等待客户端连接¶
sudo buildkitd
我这里采用 从Dockerfile构建Docker镜像 的
centos8-ssh:
CentOS 8的Dockerfile，包括ssh安装¶
# Build centos 8 image with ssh:
# ------------------------------
# docker build -f centos8-ssh -t local:centos8-ssh
# create container:
# -----------------
# docker run -itd -p 1122:22 --hostname myserver --name myserver local:centos8-ssh
# create container with volume:
# -----------------------------
# docker volume create data
# docker run -itd -p 1122:22 --hostname myserver --name myserver -v data:/data local:centos8-ssh
# create container with volume and static ip:
# -------------------------------------------
# docker volume create data
# docker network create --subnet=172.18.0.0/16 data-net
# docker run -itd -p 1122:22 --hostname myserver --name myserver -v data:/data \
# --net data-net --ip 172.18.0.252 local:centos8-ssh
# create container with volume and static ip, then limit resource and map port:
# -----------------------------------------------------------------------------
# docker volume create data
# docker network create --subnet=172.18.0.0/16 data-net
# docker run -itd -p 1122:22 --hostname myserver --name myserver -v data:/data \
# --net data-net --ip 172.18.0.252 --memory=2048M --cpus="1.5" \
# -p 22 -p 8080:80 local:centos8-ssh
FROM docker.io/centos:8
MAINTAINER vincent huatai <vincent@huatai.me>
RUN dnf clean all
RUN dnf -y update
RUN dnf -y install which sudo openssh-clients openssh-server initscripts
# Prepare sshd host key
RUN ssh-keygen -A
# add account "admin" and give sudo privilege
RUN groupadd -g 505 admin
RUN useradd -g 505 -u 505 -d /home/admin -m admin
RUN usermod -aG wheel admin
RUN echo "%wheel ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
# Add ssh public key for login
RUN mkdir -p /home/admin/.ssh
COPY authorized_keys /home/admin/.ssh/authorized_keys
RUN chown -R admin:admin /home/admin/.ssh
RUN chmod 600 /home/admin/.ssh/authorized_keys
RUN chmod 700 /home/admin/.ssh
RUN mv /var/run/nologin /var/run/nologin.bak
# run service when container started - sshd
EXPOSE 22:1122
#CMD ["/usr/sbin/sshd", "-D"]
# ----------
# WANT run sshd and get a bash
# ENTRYPOINT will not be override by commandline
# ----------
ENTRYPOINT /usr/sbin/sshd && /bin/bash
执行以下命令构建:
nerdctl build构建容器镜像¶
sudo nerdctl build -t local:centos8-ssh .
此时提示信息:
nerdctl build构建容器镜像的输出信息¶
[+] Building 53.6s (7/19)
=> [internal] load build definition from Dockerfile 0.6s
=> => transferring dockerfile: 1.00kB 0.0s
=> [internal] load .dockerignore 0.7s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/centos:8 10.7s
=> [ 1/15] FROM docker.io/library/centos:8@sha256:a27fd8080b517143cbbbab9dfb7c8571c40d67d534bbdee55bd6c473f432b177 32.7s
=> => resolve docker.io/library/centos:8@sha256:a27fd8080b517143cbbbab9dfb7c8571c40d67d534bbdee55bd6c473f432b177 0.2s
=> => sha256:a1d0c75327776413fa0db9ed3adcdbadedc95a662eb1d360dad82bb913f8a1d1 83.52MB / 83.52MB 27.0s
=> => extracting sha256:a1d0c75327776413fa0db9ed3adcdbadedc95a662eb1d360dad82bb913f8a1d1 5.0s
=> [internal] load build context 0.7s
=> => transferring context: 266B 0.0s
=> [ 2/15] RUN dnf clean all 2.1s
=> ERROR [ 3/15] RUN dnf -y update 7.3s
------
> [ 3/15] RUN dnf -y update:
#0 7.032 CentOS Linux 8 - AppStream 6.1 B/s | 38 B 00:06
#0 7.041 Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
------
Dockerfile:5
--------------------
3 |
4 | RUN dnf clean all
5 | >>> RUN dnf -y update
6 |
7 | RUN dnf -y install which sudo openssh-clients openssh-server initscripts
--------------------
error: failed to solve: process "/bin/sh -c dnf -y update" did not complete successfully: exit code: 1
FATA[0053] unrecognized image format
备注
需要注意 buildkitd.toml 配置默认的 namespace 是 k8s.io 标识针对 Kubernetes，这也是我最初不知道的要点。我刚开始还奇怪 nerdctl images 为何总是空的，原来正确方法是:
nerdctl -n k8s.io images
这样才能看到Kubernetes集群中的镜像
在上述报错信息的同时， buildkitd 控制台也输出了报错信息:
ERRO[2022-08-10T22:58:40+08:00] /moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = process "/bin/sh -c dnf -y update" did not complete successfully: exit code: 1
我以为 buildkit 默认想去连接 docker0 这个本地NAT网络，但是上文可以看到，只安装 nerdctl 只创建了 nerdctl0 命名的NAT网络。该如何配置 buildkit 去使用指定的 bridge 呢？
虽然我的节点位于Kubernetes中，并且安装了 Cilium网络 ，看起来网络配置不正确
唉，瞎猜不是好习惯，我后来才发现我根本没有仔细看报错信息 实际是Docker的centos8镜像中默认仓库配置问题: 上面提示不是已经说了是 repo 'appstream' mirrorlist错误么?
我尝试通过 sudo buildkitd --oci-worker=false --containerd-worker=true 运行 buildkitd （指定使用containerd worker) ，但是依然没有解决这个问题
但是，我注意到 buildkitd 的终端输出信息显示:
nerdctl build失败，显示使用了 systemd-resolve¶
WARN[2022-08-11T00:48:34+08:00] using host network as the default
WARN[2022-08-11T00:48:34+08:00] git source cannot be enabled: failed to find git binary: exec: "git": executable file not found in $PATH
INFO[2022-08-11T00:48:34+08:00] found worker "ndtir74093ai7v0l9y6qrxppp", labels=map[org.mobyproject.buildkit.worker.containerd.namespace:default org.mobyproject.buildkit.worker.containerd.uuid:c1b0bbf7-00fd-4fd9-8ff7-5fe4da6fa866 org.mobyproject.buildkit.worker.executor:containerd org.mobyproject.buildkit.worker.hostname:z-k8s-m-1 org.mobyproject.buildkit.worker.network:host org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/amd64/v2 linux/amd64/v3 linux/386]
INFO[2022-08-11T00:48:34+08:00] found 1 workers, default="ndtir74093ai7v0l9y6qrxppp"
WARN[2022-08-11T00:48:34+08:00] currently, only the default worker can be used.
INFO[2022-08-11T00:48:34+08:00] running server on /run/buildkit/buildkitd.sock
ERRO[2022-08-11T00:54:49+08:00] unexpected error getting ingest status of "layer-sha256:a1d0c75327776413fa0db9ed3adcdbadedc95a662eb1d360dad82bb913f8a1d1": context canceled
INFO[2022-08-11T00:54:54+08:00] detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf
ERRO[2022-08-11T00:55:02+08:00] /moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = process "/bin/sh -c dnf -y update" did not complete successfully: exit code: 1
等等，怎么会因为检测到DNS服务，就自说自话使用了 /run/systemd/resolve/resolv.conf 作为本地解析器配置？
检查了一下主机，确实运行了 systemd-resolved ，配置文件 /run/systemd/resolve/resolv.conf 内容也没有错:
nameserver 192.168.6.200
search staging.huatai.me
我在 192.168.6.200 上运行的 DNSmasq ，修订配置添加 log-queries (开启DNS查询日志)，然后重启dnsmasq，再次执行 nerdctl build ，观察DNS服务器端是否收到查询请求: 确实收到了DNS查询
CentOS 8转换CentOS Stream¶
我发现我犯了一个低级错误
#0 1.355 CentOS Linux 8 - AppStream 66 B/s | 38 B 00:00
#0 1.366 Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
参考 Error: Failed to download metadata for repo ‘appstream’: Cannot prepare internal mirrorlist: No URLs in mirrorlist 很简单，CentOS已经走到了生命尽头，需要切换到 stream 版本才能正常工作，所以Docker官方镜像已经无法直接使用
CentOS 8 reached EOL on 2021-12-31
修复的方法是采用在Dockerfile中增加切换
CentOS Stream 8:
CentOS Stream 8(取代已经 EOL CentOS 8)的Dockerfile，包括ssh安装¶
# Build centos 8 image with ssh:
# ------------------------------
# docker build -f centos8-ssh -t local:centos8-ssh
# create container:
# -----------------
# docker run -itd -p 1122:22 --hostname myserver --name myserver local:centos8-ssh
# create container with volume:
# -----------------------------
# docker volume create data
# docker run -itd -p 1122:22 --hostname myserver --name myserver -v data:/data local:centos8-ssh
# create container with volume and static ip:
# -------------------------------------------
# docker volume create data
# docker network create --subnet=172.18.0.0/16 data-net
# docker run -itd -p 1122:22 --hostname myserver --name myserver -v data:/data \
# --net data-net --ip 172.18.0.252 local:centos8-ssh
# create container with volume and static ip, then limit resource and map port:
# -----------------------------------------------------------------------------
# docker volume create data
# docker network create --subnet=172.18.0.0/16 data-net
# docker run -itd -p 1122:22 --hostname myserver --name myserver -v data:/data \
# --net data-net --ip 172.18.0.252 --memory=2048M --cpus="1.5" \
# -p 22 -p 8080:80 local:centos8-ssh
FROM centos:latest
MAINTAINER vincent huatai <vincent@huatai.me>
RUN dnf clean all
RUN dnf --disablerepo '*' --enablerepo extras swap centos-linux-repos centos-stream-repos -y
RUN dnf distro-sync -y
RUN dnf -y update
RUN dnf -y install which sudo openssh-clients openssh-server initscripts
# Prepare sshd host key
RUN ssh-keygen -A
# add account "admin" and give sudo privilege
RUN groupadd -g 505 admin
RUN useradd -g 505 -u 505 -d /home/admin -m admin
RUN usermod -aG wheel admin
RUN echo "%wheel ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
# Add ssh public key for login
RUN mkdir -p /home/admin/.ssh
COPY authorized_keys /home/admin/.ssh/authorized_keys
RUN chown -R admin:admin /home/admin/.ssh
RUN chmod 600 /home/admin/.ssh/authorized_keys
RUN chmod 700 /home/admin/.ssh
RUN mv /var/run/nologin /var/run/nologin.bak
# run service when container started - sshd
EXPOSE 22:1122
#CMD ["/usr/sbin/sshd", "-D"]
# ----------
# WANT run sshd and get a bash
# ENTRYPOINT will not be override by commandline
# ----------
ENTRYPOINT /usr/sbin/sshd && /bin/bash
然后执行:
nerdctl build构建容器镜像centos-stream-8-ssh¶
sudo nerdctl build -t local:centos-stream-8-ssh .
然后运行:
sudo nerdctl run -it -p 1122:22 --hostname centos-stream-8 --name centos-stream-8 local:centos-stream-8-ssh
现在就可以ssh进入容器了:
ssh admin@192.168.6.101 -p 1122
但是，使用 nerdctl run -it 命令启动，不能退出，一旦退出容器也就终止了。此时再次执行 nerdctl start centos-stream-8 会报错:
failed to create shim task: OCI runtime create failed: runc create failed: cannot allocate tty if runc will detach without setting console socket: unknown
实际上 nerdctl 有一个参数 -d 可以启动时直接放到后台运行:
sudo nerdctl run -d -p 1122:22 --hostname centos-stream-8 --name centos-stream-8 local:centos-stream-8-ssh
nerdctl加载镜像到Kubernetes¶
nerdctl 可以直接为本地Kubernetes构建镜像而无需registry:
这里采用 Containerd环境CentOS容器运行systemd 实践验证通过的
Dockerfile:
CentOS Stream 8 启用systemd(正确)¶
# USE DOCKER
# docker build --rm -t local/centos-stream-8-systemd .
# docker run -itd --privileged=true -p 1122:22 --hostname myserver --name myserver local:centos-stream-8-systemd
# USE nerdctl (containerd)
# nerdctl build -t centos-stream-8-systemd .
# nerdctl run -it --privileged=true -p 1122:22 --hostname centos-stream-8 --name centos-stream-8 centos-stream-8-systemd:latest
# nerdctl run -d --privileged=true -p 1122:22 --hostname centos-stream-8 --name centos-stream-8 centos-stream-8-systemd:latest
FROM centos:latest
MAINTAINER vincent huatai <vincent@huatai.me>
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
#VOLUME [ "/sys/fs/cgroup" ]
RUN --mount=type=bind,target=/sys/fs/cgroup \
--mount=type=bind,target=/sys/fs/fuse \
--mount=type=tmpfs,target=/tmp \
--mount=type=tmpfs,target=/run \
--mount=type=tmpfs,target=/run/lock
RUN dnf clean all
RUN dnf --disablerepo '*' --enablerepo extras swap centos-linux-repos centos-stream-repos -y
RUN dnf distro-sync -y
RUN dnf -y update
RUN dnf -y install which sudo openssh-clients openssh-server initscripts
# Prepare sshd host key
RUN ssh-keygen -A
# add account "admin" and give sudo privilege
RUN groupadd -g 505 admin
RUN useradd -g 505 -u 505 -d /home/admin -m admin
RUN usermod -aG wheel admin
RUN echo "%wheel ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
# Add ssh public key for login
RUN mkdir -p /home/admin/.ssh
COPY authorized_keys /home/admin/.ssh/authorized_keys
RUN chown -R admin:admin /home/admin/.ssh
RUN chmod 600 /home/admin/.ssh/authorized_keys
RUN chmod 700 /home/admin/.ssh
RUN mv /var/run/nologin /var/run/nologin.bak
# run service when container started - sshd
EXPOSE 22:1122
# systemd
# CMD ["/usr/sbin/init"]
ENTRYPOINT [ "/usr/lib/systemd/systemd" ]
CMD [ "log-level=info", "unit=sysinit.target" ]
执行以下命令为Kuberntes构建镜像:
nerdctl build为本地Kubernetes构建镜像¶
nerdctl --namespace k8s.io build -t centos-stream-8-systemd .
现在通过
nerdctl命令可以检查到新生成到镜像:sudo nerdctl -n k8s.io images
显示:
REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE
...
centos-stream-8-systemd latest 758665d16a77 5 minutes ago linux/amd64 705.0 MiB 282.0 MiB
...
然后在Kubernetes集群生成pod:
执行kubeclt apply 将 nerdctl build为本地Kubernetes构建镜像 生成可运行pod¶
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: centos-stream-8
spec:
containers:
- name: centos-stream-8
image: centos-stream-8-systemd
imagePullPolicy: Never
securityContext:
privileged: true
EOF
备注
此时提示:
pod/centos-stream-8 created
但是，此时检查并不一定会看到pod运行了:
kubectl get pods
显示:
NAME READY STATUS RESTARTS AGE
centos-stream-8 0/1 ErrImageNeverPull 0 3m7s
这是因为没有使用 Kubernetes部署registry仓库 ，需要将镜像导入到每个调度到的节点上才能启动。
检查新创建的pod调度到哪个节点:
$ kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES centos-stream-8 0/1 ErrImageNeverPull 0 4m41s 10.0.4.224 z-k8s-n-2 <none> <none>
我们需要使用类似 docker save 的命令来导出镜像，复制到该节点:
sudo nerdctl save centos-stream-8-systemd > centos-stream-8-systemd.tar
然后在目标节点
z-k8s-n-2上导入这个镜像:sudo nerdctl -n k8s.io load < centos-stream-8-systemd.tar
备注
nerdctl 的 save 和 load 完全和 docker 的 无需Docker Registry传输Docker镜像 兼容，相互之间可以转换传输。此外，命令格式其实也一样，例如可以使用和 docker 一样的参数保存镜像和导入镜像:
nerdctl保存和加载镜像命令¶
nerdctl save -o ~/x-dev.tar local:x-dev
nerdctl load -i ~/x-dev.tar
完成后再次检查集群:
kubectl get pods -o wide
就可以看到pod容器正常运行了:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
centos-stream-8 1/1 Running 0 17m 10.0.4.224 z-k8s-n-2 <none> <none>
不过，此时运行的容器网络还没有配置好，所以从外部还不能 ssh 登陆这个容器。但是，容器内部是可以访问外部的，例如，我们可以通过 kubectl exec 登陆到容器的内部检查:
kubectl exec -it centos-stream-8 -- /bin/bash
检查容器内部:
[root@centos-stream-8 /]# df -h
Filesystem Size Used Avail Use% Mounted on
overlay 9.4G 2.3G 7.1G 24% /
tmpfs 64M 0 64M 0% /dev
/dev/vda2 6.3G 4.2G 2.2G 67% /etc/hosts
/dev/vdb1 9.4G 2.3G 7.1G 24% /etc/hostname
shm 64M 0 64M 0% /dev/shm
tmpfs 2.0G 8.1M 2.0G 1% /run
[root@centos-stream-8 /]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
14: eth0@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 4a:a2:3f:b5:da:ae brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.0.4.224/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::48a2:3fff:feb5:daae/64 scope link
valid_lft forever preferred_lft forever
可以在内部安装软件包:
dnf install net-tools -y
检查路由:
# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.4.25 0.0.0.0 UG 0 0 0 eth0
10.0.4.25 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
检查容器内部进程:
top
显示:
top - 22:48:00 up 37 days, 47 min, 0 users, load average: 0.02, 0.03, 0.00
Tasks: 6 total, 1 running, 5 sleeping, 0 stopped, 0 zombie
%Cpu(s): 2.3 us, 1.0 sy, 0.0 ni, 96.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 3929.8 total, 537.3 free, 438.8 used, 2953.6 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 3191.6 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 90104 10248 8236 S 0.0 0.3 0:00.21 systemd
28 root 20 0 87388 8360 7508 S 0.0 0.2 0:00.13 systemd-journal
35 dbus 20 0 54108 4124 3696 S 0.0 0.1 0:00.00 dbus-daemon
36 root 20 0 76628 7280 6372 S 0.0 0.2 0:00.00 sshd
39 root 20 0 15104 3824 3252 S 0.0 0.1 0:00.07 bash
103 root 20 0 52080 4360 3712 R 0.0 0.1 0:00.00 top
接下来，我们需要配置 Ingress 对外暴露SSH服务，在 Cilium网络 中， 内置的 Cilium Kubernetes Ingress 采用 Envoy 实现
也可以采用多种 Ingress控制器(controller) ，例如我准备部署:
简单的NodePort输出验证¶
备注
参考 Cilium完全取代kube-proxy运行Kubernetes 验证方法，将ssh服务输出到集群
首先为前面创建的
centos-stream-8添加标签:kubectl label pods centos-stream-8 run=dev
添加了 run=dev 标签之后，就可以配置服务
现在我们检查 run=dev 标签的pods:
kubectl get pods -l run=dev -o wide
显示:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
centos-stream-8 1/1 Running 0 2d2h 10.0.4.224 z-k8s-n-2 <none> <none>
为实例创建NodePort Kubeernetes服务(services)
kubectl expose pods centos-stream-8 --type=NodePort --port=22
此时提示:
service/centos-stream-8 exposed
注意如果没有标签的pods是无法输出，会报错(这也是为何前面需要先给pod添加标签的原因，因为kubernetes输出服务端口是根据labels确定的):
error: couldn't retrieve selectors via --selector flag or introspection: the pod has no labels and cannot be exposed
检查输出的NodePort服务:
kubectl get svc centos-stream-8
显示:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
centos-stream-8 NodePort 10.111.120.89 <none> 22:30134/TCP 106s
同样，我们再次检查
cilium service listkubectl -n kube-system exec ds/cilium -- cilium service list
输出就有:
...
16 10.111.120.89:22 ClusterIP 1 => 10.0.4.224:22
17 192.168.6.102:30134 NodePort 1 => 10.0.4.224:22
18 0.0.0.0:30134 NodePort 1 => 10.0.4.224:22
现在我们就可以访问:
ssh admin@192.168.6.102 -p 30134
登陆到容器内部进行运维