1 、擦除了磁盘分区以及标签等 wipefs -a -f /dev/sdc ,并使用 blkid 查看磁盘信息,已验证磁盘信息都已清除。
2 、怀疑硬盘是否损坏,对磁盘进行分区格式化并挂载验证查验能正常写入和读取数据,磁盘正常,之后重新进行#1 ,问题依旧
执行 ceph orch daemon add osd ceph-2:/dev/sdc 报错,说是什么权限不行,可是我另外两台一样的操作,一样的 add osd 都是可以的,用户使用的 root 。报错如下:
Error EINVAL: Traceback (most recent call last):
File "/usr/share/ceph/mgr/mgr_module.py", line 1809, in _handle_command
return self.handle_command(inbuf, cmd)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 183, in handle_command
return dispatch[cmd['prefix']].call(self, cmd, inbuf)
File "/usr/share/ceph/mgr/mgr_module.py", line 474, in call
return self.func(mgr, **kwargs)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 119, in
wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs) # noqa: E731
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 108, in wrapper
return func(*args, **kwargs)
File "/usr/share/ceph/mgr/orchestrator/module.py", line 1100, in _daemon_add_osd
raise_if_exception(completion)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 240, in raise_if_exception
raise e
RuntimeError: cephadm exited with an error code: 1, stderr:Inferring config /var/lib/ceph/fd4f9c76-0c43-11ef-a0ee-37aaf28eb433/mon.ceph-2/config
Non-zero exit code 1 from /usr/bin/docker run --rm --ipc=host --stop-signal=SIGTERM --ulimit nofile=1048576 --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.io/ceph/ceph@sha256:d55fa1ab2bf753fab11fdd9f1bb9106ee5aecccdd0e9532ef0125bead1adf3c9 -e NODE_NAME=ceph-2 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_OSDSPEC_AFFINITY=None -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/fd4f9c76-0c43-11ef-a0ee-37aaf28eb433:/var/run/ceph:z -v /var/log/ceph/fd4f9c76-0c43-11ef-a0ee-37aaf28eb433:/var/log/ceph:z -v /var/lib/ceph/fd4f9c76-0c43-11ef-a0ee-37aaf28eb433/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /:/rootfs -v /tmp/ceph-tmp92x1hz9n:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmptnkwq47u:/var/lib/ceph/bootstrap-osd/ceph.keyring:z quay.io/ceph/ceph@sha256:d55fa1ab2bf753fab11fdd9f1bb9106ee5aecccdd0e9532ef0125bead1adf3c9 lvm batch --no-auto /dev/sdc --yes --no-systemd
/usr/bin/docker: stderr --> passed data devices: 1 physical, 0 LVM
/usr/bin/docker: stderr --> relative data size: 1.0
/usr/bin/docker: stderr Running command: /usr/bin/ceph-authtool --gen-print-key
/usr/bin/docker: stderr Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new be074625-76ae-4343-a080-0b4a7df31bde
/usr/bin/docker: stderr Running command: nsenter --mount=/rootfs/proc/1/ns/mnt --ipc=/rootfs/proc/1/ns/ipc --net=/rootfs/proc/1/ns/net --uts=/rootfs/proc/1/ns/uts /sbin/vgcreate --force --yes ceph-fd0addc7-e222-4413-850d-ff99e750a4f9 /dev/sdc
/usr/bin/docker: stderr stdout: Physical volume "/dev/sdc" successfully created.
/usr/bin/docker: stderr stdout: Volume group "ceph-fd0addc7-e222-4413-850d-ff99e750a4f9" successfully created
/usr/bin/docker: stderr Running command: nsenter --mount=/rootfs/proc/1/ns/mnt --ipc=/rootfs/proc/1/ns/ipc --net=/rootfs/proc/1/ns/net --uts=/rootfs/proc/1/ns/uts /sbin/lvcreate --yes -l 142812 -n osd-block-be074625-76ae-4343-a080-0b4a7df31bde ceph-fd0addc7-e222-4413-850d-ff99e750a4f9
/usr/bin/docker: stderr stdout: Wiping xfs signature on /dev/ceph-fd0addc7-e222-4413-850d-ff99e750a4f9/osd-block-be074625-76ae-4343-a080-0b4a7df31bde.
/usr/bin/docker: stderr stdout: Logical volume "osd-block-be074625-76ae-4343-a080-0b4a7df31bde" created.
/usr/bin/docker: stderr Running command: /usr/bin/ceph-authtool --gen-print-key
/usr/bin/docker: stderr Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-2
/usr/bin/docker: stderr Running command: /usr/bin/chown -h ceph:ceph /dev/ceph-fd0addc7-e222-4413-850d-ff99e750a4f9/osd-block-be074625-76ae-4343-a080-0b4a7df31bde
/usr/bin/docker: stderr Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1
/usr/bin/docker: stderr Running command: /usr/bin/ln -s /dev/ceph-fd0addc7-e222-4413-850d-ff99e750a4f9/osd-block-be074625-76ae-4343-a080-0b4a7df31bde /var/lib/ceph/osd/ceph-2/block
/usr/bin/docker: stderr Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-2/activate.monmap
/usr/bin/docker: stderr stderr: got monmap epoch 3
/usr/bin/docker: stderr --> Creating keyring file for osd.2
/usr/bin/docker: stderr Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2/keyring
/usr/bin/docker: stderr Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2/
/usr/bin/docker: stderr Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 2 --monmap /var/lib/ceph/osd/ceph-2/activate.monmap --keyfile - --osdspec-affinity None --osd-data /var/lib/ceph/osd/ceph-2/ --osd-uuid be074625-76ae-4343-a080-0b4a7df31bde --setuser ceph --setgroup ceph
/usr/bin/docker: stderr stderr: 2024-05-08T01:49:19.207+0000 7f21171bf640 -1 bluestore(/var/lib/ceph/osd/ceph-2//block) _read_bdev_label unable to decode label at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
/usr/bin/docker: stderr stderr: 2024-05-08T01:49:19.207+0000 7f21171bf640 -1 bluestore(/var/lib/ceph/osd/ceph-2//block) _read_bdev_label unable to decode label at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
/usr/bin/docker: stderr stderr: 2024-05-08T01:49:19.207+0000 7f21171bf640 -1 bluestore(/var/lib/ceph/osd/ceph-2//block) _read_bdev_label unable to decode label at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
/usr/bin/docker: stderr stderr: 2024-05-08T01:49:19.207+0000 7f21171bf640 -1 bluestore(/var/lib/ceph/osd/ceph-2/) _read_fsid unparsable uuid
/usr/bin/docker: stderr stderr: 2024-05-08T01:49:19.735+0000 7f21171bf640 -1 bluestore(/var/lib/ceph/osd/ceph-2//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2//block: (13) Permission denied
/usr/bin/docker: stderr stderr: 2024-05-08T01:49:19.735+0000 7f21171bf640 -1 bluestore(/var/lib/ceph/osd/ceph-2//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2//block: (13) Permission denied
/usr/bin/docker: stderr stderr: 2024-05-08T01:49:19.735+0000 7f21171bf640 -1 bluestore(/var/lib/ceph/osd/ceph-2//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2//block: (13) Permission denied
/usr/bin/docker: stderr stderr: 2024-05-08T01:49:19.735+0000 7f21171bf640 -1 bluestore(/var/lib/ceph/osd/ceph-2//block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2//block: (13) Permission denied