在Linux Jail中使用EXT文件系统

在构建 使用 Rocky-Container-base tgz 包部署Linux Jail Rocky 之后，我尝试将 FreeBSD使用Linux XFS文件系统 用于 LFS分区(和FreeBSD一起Dualboot) ，但是实践发现采用FUSE方式访问Linux文件系统限制较多且不稳定。所以我改为采用 FreeBSD使用Linux EXT文件系统 来构建一个较为稳定的Linux文件系统，这个步骤完成后可以在Host主机上看到分区挂载:

分区挂载

/dev/diskid/DISK-Y39B70RTK7ASp6                197G    1.0G    186G     1%    /lfs

备注

在FreeBSD观察到的分区使用情况和在Linux虚拟机中观察的分区使用情况不同。上述观察到分区使用1G空间，在虚拟机观察挂载的磁盘只使用了 2.1M

启动Linux容器时挂载Host主机EXT4目录

Jail容器不是Linux虚拟机，实际上依然是FreeBSD系统，无法直接使用 XFS文件系统 ，需要通过 nullfs 来挂载Host主机已经通过FUSE挂载的XFS分区。所以修订 /etc/jail.conf.d/lrdev.conf ，进一步添加自动挂载和写在Nullfs的配置命令:

调整 /etc/jail.conf.d/lrdev.conf 增加Nullfs绑定 EXT4分区

lrdev {
  # thin jail devfs_ruleset 5 和Linux Jail的4不同
  devfs_ruleset=4;

  # HOSTNAME/PATH - Snapshot
  path = "/zdata/jails/containers/${name}";

  # NETWORKS/INTERFACES
  $id = "253";
  $ip = "192.168.7.${id}/24";

  # MOUNT
  mount += "devfs     $path/compat/rocky/dev     devfs     rw  0 0";
  mount += "tmpfs     $path/compat/rocky/dev/shm tmpfs     rw,size=1g,mode=1777  0 0";
  mount += "fdescfs   $path/compat/rocky/dev/fd  fdescfs   rw,linrdlnk 0 0";
  mount += "linprocfs $path/compat/rocky/proc    linprocfs rw  0 0";
  mount += "linsysfs  $path/compat/rocky/sys     linsysfs  rw  0 0";
  mount += "/tmp      $path/compat/rocky/tmp     nullfs    rw  0 0";
  mount += "/home     $path/compat/rocky/home    nullfs    rw  0 0";

  # MOUNT EXT4
  exec.poststart += "mount -t nullfs /lfs  $path/compat/rocky/ext4_lfs";
  exec.poststop  += "umount  $path/compat/rocky/ext4_lfs";

}

上述容器启动和停止时XFS目录挂载和卸载也可以改写成

而公共配置部分不需要调整，保留 /etc/jail.conf 不变:

混合多种jail的公共 /etc/jail.conf

# STARTUP/LOGGING
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.consolelog = "/var/log/jail_console_${name}.log";

# PERMISSIONS
allow.raw_sockets;
exec.clean;
mount.devfs;

allow.mount;
allow.mount.devfs;

enforce_statfs = 1;

# HOSTNAME
host.hostname = "${name}";

# NETWORK - VNET/VIMAGE
#ip4 = inherit;
interface = igc0bridge;
vnet;
vnet.interface = "${epair}b";
# common NETWORK config
$gateway = "192.168.7.221";
$bridge = "igc0bridge";
$epair = "epair${id}";

# ADD TO bridge INTERFACE
exec.prestart += "ifconfig ${epair} create up";
exec.prestart += "ifconfig ${epair}a up descr jail:${name}";
exec.prestart += "ifconfig ${bridge} addm ${epair}a up";
exec.start    += "ifconfig ${epair}b ${ip} up";
exec.start    += "route add default ${gateway}";
exec.poststop = "ifconfig ${bridge} deletem ${epair}a";
exec.poststop += "ifconfig ${epair}a destroy";

.include "/etc/jail.conf.d/*.conf";

在启动 lrdev 之前，需要确保Linux compact chroot 目录下存在 /zdata/jails/containers/${name}/compat/rocky/ext4_lfs 目录:

创建目录

# 按照 /etc/jail.conf.d/lrdev.conf 配置创建目录
name=lrdev
path="/zdata/jails/containers/${name}"
mount_dir="$path/compat/rocky/ext4_lfs"

mkdir $mount_dir

启动容器 lrdev :

启动 lrdev Linux Jail

service jail start lrdev

• 进入 lrdev 的Linux环境:

进入 lrdev 的Linux环境

jexec lrdev chroot /compat/rocky /bin/bash

此时在容器内部就可以访问Host主机挂载的 FreeBSD使用Linux EXT文件系统

采用fstab方式挂载nullfs

上述配置文件也可以修订成独立的 fstab 以便启动容器时挂载多个目录

独立的 fstab 配置文件 /zdata/jails/lrdev-nullfs.fstab

/lfs    /zdata/jails/containers/lrdev/compat/rocky/ext4_lfs    nullfs    rw  0  0

然后修订 /etc/jail.conf.d/lrdev.conf 引用 /zdata/jails/lrdev-nullfs.fstab

修订 /etc/jail.conf.d/lrdev.conf 引用 /zdata/jails/lrdev-nullfs.fstab

lrdev {
  # thin jail devfs_ruleset 5 和Linux Jail的4不同
  devfs_ruleset=4;

  # HOSTNAME/PATH - Snapshot
  path = "/zdata/jails/containers/${name}";

  # NETWORKS/INTERFACES
  $id = "253";
  $ip = "192.168.7.${id}/24";

  # MOUNT
  mount += "devfs     $path/compat/rocky/dev     devfs     rw  0 0";
  mount += "tmpfs     $path/compat/rocky/dev/shm tmpfs     rw,size=1g,mode=1777  0 0";
  mount += "fdescfs   $path/compat/rocky/dev/fd  fdescfs   rw,linrdlnk 0 0";
  mount += "linprocfs $path/compat/rocky/proc    linprocfs rw  0 0";
  mount += "linsysfs  $path/compat/rocky/sys     linsysfs  rw  0 0";
  mount += "/tmp      $path/compat/rocky/tmp     nullfs    rw  0 0";
  mount += "/home     $path/compat/rocky/home    nullfs    rw  0 0";

  # MOUNT EXT4
  #exec.poststart += "mount -t nullfs /lfs  $path/compat/rocky/ext4_lfs";
  #exec.poststop  += "umount  $path/compat/rocky/ext4_lfs";
  mount.fstab = "/zdata/jails/${name}-nullfs.fstab";

}

采用单行配置nullfs目录挂载

不过，对于少量目录挂载，其实单条mount配置就可以，上述配置简化为:

单条mount配置挂载nullfs /etc/jail.conf.d/lrdev.conf

lrdev {
  # thin jail devfs_ruleset 5 和Linux Jail的4不同
  devfs_ruleset=4;

  # HOSTNAME/PATH - Snapshot
  path = "/zdata/jails/containers/${name}";

  # NETWORKS/INTERFACES
  $id = "253";
  $ip = "192.168.7.${id}/24";

  # MOUNT
  mount += "devfs     $path/compat/rocky/dev     devfs     rw  0 0";
  mount += "tmpfs     $path/compat/rocky/dev/shm tmpfs     rw,size=1g,mode=1777  0 0";
  mount += "fdescfs   $path/compat/rocky/dev/fd  fdescfs   rw,linrdlnk 0 0";
  mount += "linprocfs $path/compat/rocky/proc    linprocfs rw  0 0";
  mount += "linsysfs  $path/compat/rocky/sys     linsysfs  rw  0 0";
  mount += "/tmp      $path/compat/rocky/tmp     nullfs    rw  0 0";
  mount += "/home     $path/compat/rocky/home    nullfs    rw  0 0";

  # MOUNT EXT4 /lfs
  mount += "/lfs     $path/compat/rocky/ext4_lfs    nullfs    rw  0 0";


}

验证

在Jail lrdev 容器内检查 /ext4_lfs 目录，并复制文件或编辑文件，然后在Host主机上对应的 lfs 能够看到同步变化

下一步

• LFS准备工作