LVM执行 pvcreate 时报错 Device /dev/XXX excluded by a filter.

在部署 Linux LVM逻辑卷管理 时，有时候我们会在执行 pvcreate /dev/XXX 时候遇到警示:

Device /dev/XXX excluded by a filter.

这个报错(我遇到过)通常有两种情况:

  • 使用的磁盘是曾经创建过文件系统分区表的磁盘， lvm 卷管理程序会忽略已经创建过分区表的磁盘分区

  • 存储设备驱动特殊，识别为特殊的设备名， lvm 默认只支持在常见设备，如 /dev/sdX , /dev/vdX ，对于特殊设备需要修订 lvm.conf 增加支持类型

实践案例

在生产环境 扩展LVM上的EXT4文件系统 遇到一个困境:

  • 对一块已经使用过的磁盘， 部署LVM 执行:

执行 pvcreate 创建物理卷
pvcreate /dev/dfb1

此时会报错:

执行 pvcreate 创建物理卷报错
Device /dev/dfb1 excluded by a filter.

  • 这个报错我以前也遇到过，也就是我上文说的第一种情况。一般可以参考 Device /dev/sdX excluded by a filter. 通过 wipefs 抹去旧磁盘分区表来修正:

执行 wipefs 抹去旧磁盘分区表
wipefs -a /dev/dfb

提示信息:

执行 wipefs 抹去旧磁盘分区表时输出信息
/dev/dfb: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/dfb: 8 bytes were erased at offset 0x5d21dbffe00 (gpt): 45 46 49 20 50 41 52 54
/dev/dfb: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/dfb: calling ioclt to re-read partition table: Success

我以为已经解决了问题，所以重新再使用 parted分区工具 进行分区，然后再次执行 pvcreate /dev/dfb1

WHAT?

报错依旧:

执行 pvcreate 创建物理卷报错
Device /dev/dfb1 excluded by a filter.

反复几次，我怀疑是 wipefs 仅抹去了磁盘开头的分区表，但是没有破坏分区表后面的文件系统?

  • 所以我又尝试通过 dd 命令多抹去一些数据(磁盘开头的2MB):

通过 dd 抹去磁盘开头2MB数据，试图彻底抹去分区表和文件系统
dd if=/dev/zero of=/dev/dfb bs=1M count=2

  • 再来一遍，报错依旧

排查

  • 执行 pvcreate 的debug模式:

使用 -vvv 参数详细调试 pvcreate
pvcreate /dev/dfb1 -vvv

此时会输出详细调试信息，注意观察出现报错前后的信息:

使用 -vvv 参数详细调试 pvcreate 输出信息
        Parsing: pvcreate /dev/dfb1 -vvv
        Recognised command pvcreate_general (id 100 / enum 80).
      devices/global_filter not found in config: defaulting to global_filter = [ "a|.*/|" ]
      global/lvmetad_update_wait_time not found in config: defaulting to 10
        /run/lvm/lvmetad.socket: Opening daemon socket to lvmetad for protocol lvmetad version 1.
...
        /dev/dfb1: Skipping: Unrecognised LVM device type 253
...
        Found dev 253:0 /dev/dfa - exists.
        Found dev 253:0 /dev/disk/by-id/pcie-aliflash-SF16308K7520272 - exists.
        Found dev 253:1 /dev/dfa1 - exists.
        Found dev 253:1 /dev/disk/by-id/pcie-aliflash-SF16308K7520272-part1 - exists.
        Found dev 253:1 /dev/disk/by-label/\x2fdisk1 - exists.
        Found dev 253:1 /dev/disk/by-partuuid/4fad7daa-ac98-4d6d-a806-22d535296679 - exists.
        Found dev 253:1 /dev/disk/by-uuid/721ba09c-c646-4ea0-868a-7fb70048b452 - exists.
        Found dev 253:64 /dev/dfb - exists.
        Found dev 253:64 /dev/disk/by-id/pcie-aliflash-SF16308K7520294 - exists.
        Found dev 253:65 /dev/dfb1 - exists.
        Found dev 253:65 /dev/disk/by-id/pcie-aliflash-SF16308K7520294-part1 - exists.
        Found dev 253:65 /dev/disk/by-partlabel/lvm-data - exists.
        Found dev 253:65 /dev/disk/by-partuuid/c58e465b-aa25-4a5f-beb4-ee4bf42af455 - exists.
...
        /dev/dfb: Skipping: Unrecognised LVM device type 253
        /dev/dfb1: Skipping: Unrecognised LVM device type 253
...
  Device /dev/dfb1 excluded by a filter.
...

奇怪， device type 253 是什么鬼? 为何不能识别而跳过?

  • 依次执行一些lvm检查命令进行排查:

排查lvm
lvm version
lvm dumpconfig devices

  • 关键命令，通过 /proc/devices 设备识别:

通过 /proc/devices 可以识别出设备名以及对应设备号
cat /proc/devices

从设备输出中果然可以找到编号为 253 的设备:

通过 /proc/devices 可以找到 id 是 253 的设备
Character devices:
...
Block devices:
259 blkext
  8 sd
  9 md
...
252 vrbd
253 aliflash
254 mdp

明白了，原来我们服务器使用了 aliflash 存储设备(国产化设备)

LVM默认只配置了常规的磁盘设备名，我们服务器上使用了特殊的 aliflash 设备，这个设备被识别为 /dev/dfX ，需要增加到 /etc/lvm/lvm.conf 中设备类型

  • 检查磁盘设备:

检查磁盘设备
ls -lh /dev/df*

可以看到 dfX 设备的id确实是 253 (第5列):

检查磁盘设备
brw-rw---- 1 root disk 253,  0 Jul 29 14:59 /dev/dfa
brw-rw---- 1 root disk 253,  1 Jul 29 14:59 /dev/dfa1
brw-rw---- 1 root disk 253, 64 Jul 29 14:59 /dev/dfb
brw-rw---- 1 root disk 253, 65 Jul 29 14:59 /dev/dfb1

修正

修改 /etc/lvm/lvm.conf 添加:

types = [ "aliflash", 16 ]

这样就能够匹配使用 /dev/dfa 和 /dev/dfb 等设备(注意，不是配置 df ，而是配置 /proc/devices 中设备名 aliflash 也就是对应报错中 253 id的设备名)

然后执行:

lvmdiskscan

就会看到新增加了识别出可用的设备:

/dev/sda1 [       3.00 MiB]
/dev/dfa1 [       5.82 TiB]
/dev/sda2 [       1.00 GiB]
/dev/sda3 [      50.00 GiB]
/dev/sda4 [       2.00 GiB]
/dev/sda5 [    <263.72 GiB]
/dev/dfb1 [       5.82 TiB]
0 disks
7 partitions
0 LVM physical volume whole disks
0 LVM physical volumes

现在就可以为 /dev/dfb1 和 /dev/dfa1 添加 LVM 标记了

再次执行:

pvcreate /dev/dfb1

就会提示成功:

Physical volume "/dev/dfb1" successfully created.

  • 后面就可以正常使用:

    vgcreate vg-data /dev/dfb1

提示:

Volume group "vg-data" successfully created

其他检查:

#vgdisplay
  --- Volume group ---
  VG Name               vg-data
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               5.82 TiB
  PE Size               4.00 MiB
  Total PE              1525878
  Alloc PE / Size       0 / 0
  Free  PE / Size       1525878 / 5.82 TiB
  VG UUID               VnvQM9-hcX6-gVqL-Nlsl-q7GP-CIC6-Dwbx6n

  • 创建LVM:

    lvcreate -n lv-thanos -L 5.82T vg-data

提示:

Rounding up size to full physical extent 5.82 TiB
Logical volume "lv-thanos" created.

  • 检查lvm:

    #lvdisplay
  --- Logical volume ---
  LV Path                /dev/vg-data/lv-thanos
  LV Name                lv-thanos
  VG Name                vg-data
  LV UUID                ELdzbI-Jdg8-5N1L-0S3C-tXxO-lvm9-abHl3n
  LV Write Access        read/write
  LV Creation host, time alipaydockerphy010052095245.et15, 2023-07-29 15:33:17 +0800
  LV Status              available
  # open                 0
  LV Size                5.82 TiB
  Current LE             1525679
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           251:0

#vgdisplay
  --- Volume group ---
  VG Name               vg-data
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               5.82 TiB
  PE Size               4.00 MiB
  Total PE              1525878
  Alloc PE / Size       1525679 / 5.82 TiB
  Free  PE / Size       199 / 796.00 MiB
  VG UUID               VnvQM9-hcX6-gVqL-Nlsl-q7GP-CIC6-Dwbx6n

看来还是不要指定 lv 大小，最好能够百分百

改为:

lvcreate -n lv-thanos -l 100%FREE vg-data

则提示是完整的:

Logical volume "lv-thanos" created.

此时看vg-data已经全部分配完:

#vgdisplay
  --- Volume group ---
  VG Name               vg-data
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               5.82 TiB
  PE Size               4.00 MiB
  Total PE              1525878
  Alloc PE / Size       1525878 / 5.82 TiB
  Free  PE / Size       0 / 0
  VG UUID               VnvQM9-hcX6-gVqL-Nlsl-q7GP-CIC6-Dwbx6n

参考