openEuler创建iSCSI服务 iSCSI(Internet Small Computer System Interface,发音为/ˈаɪskʌzi/,Internet小型计算机系统接口),又称为IP-SAN,是一种基于因特网及SCSI-3协议下的存储技术,由IETF提出,并于2003年2月11日成为正式的标准。
与传统的SCSI技术比较起来,iSCSI技术有以下三个革命性的变化:
把原来只用于本机的SCSI协议透过TCP/IP网络发送,使连接距离可作无限的地域延伸;
连接的服务器数量无限(原来的SCSI-3的上限是15);
由于是服务器架构,因此也可以实现在线扩容以至动态部署。 iSCSI是由IBM发明的基于以太网的存储协议,该协议与SUN的NFS协议都是为了解决存储资源共享问题的解决方案。两者意图一致,只不过两者是不同的实现方式,前者在客户机上呈现的是一个块设备,而后者则是一个目录树。在前面已经在CentOS实现nft和cifs共享案例,所以这次,我们尝试使用openEuler来部署iSCSI服务。
iSCSI架构及搭建 iSCSI架构大概如下所示:
1 2 3 4 5 +----------------------+ +----------------------+ | [ iSCSI Target ] |192.168.31.150 | 192.168.31.165| [ iSCSI Initiator ] | | server.ywmy.xyz +---------------+---------------+ client.ywmy.xyz | | | | | +----------------------+ +----------------------+
我们需要两台电脑,一台做为服务器,上面挂载磁盘,部署iSCSI Target,另外一台作为iSCSI Initiator,挂载来自服务端共享的磁盘。
服务端配置 搭建的目的,通过iSCSI实现把服务器的磁盘块,和文件映射为电脑的本地磁盘。所以我们需要给服务器添加一块磁盘,10GB大小,用于部署需求。
以磁盘形式添加 目前系统的磁盘结构如下:
1 2 3 4 5 6 7 8 9 [root@ha1 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 20G 0 disk ├─sda1 8:1 0 1M 0 part ├─sda2 8:2 0 1G 0 part /boot └─sda3 8:3 0 19G 0 part ├─openeuler-root 253:0 0 17G 0 lvm / └─openeuler-swap 253:1 0 2G 0 lvm [SWAP] sdb 8:16 0 10G 0 disk
添加的sdb即为准备分配给客户端的磁盘。
以文件形式添加
我们使用命令在/opt目录下生成镜像文件,大小一GB,
1 dd if =/dev/zero of=/opt/disk.img bs=1024k count=1024
target安装
安装target openEuler和CentOS一样,我们直接从源里安装target即可。
1 2 3 4 5 6 7 8 9 10 11 12 yum install targetcli systemctl start target systemctl enable target systemctl status target ● target.service - Restore LIO kernel target configuration Loaded: loaded (/usr/lib/systemd/system/target.service; enabled; preset: disabled) Active: active (exited) since Tue 2025-01-07 11:54:44 CST; 2h 1min ago Process: 900 ExecStart=/usr/bin/targetctl restore (code=exited, status=0/SUCCESS) Main PID: 900 (code=exited, status=0/SUCCESS) 1月 07 11:54:42 ha1 systemd[1]: Starting Restore LIO kernel target configuration... 1月 07 11:54:44 ha1 systemd[1]: Finished Restore LIO kernel target configuration.
防火墙放通iSCSI服务端口
1 2 firewall-cmd --permanent --add-port=3260/tcp firewall-cmd --complete-reload
target配置 输入targetcli进入配置。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [root@ha1 ~]# targetcli targetcli shell version 2.1.56 Copyright 2011-2013 by Datera, Inc and others. For help on commands, type 'help' . /> ls o- / .................................................................................................................... [...] o- backstores ......................................................................................................... [...] | o- block ............................................................................................. [Storage Objects: 1] | o- fileio ............................................................................................ [Storage Objects: 1] | o- pscsi ............................................................................................. [Storage Objects: 0] | o- ramdisk ........................................................................................... [Storage Objects: 0] o- iscsi ....................................................................................................... [Targets: 1] o- loopback .................................................................................................... [Targets: 0] o- vhost ....................................................................................................... [Targets: 0] o- xen-pvscsi .................................................................................................. [Targets: 0] />
通过targetcli调用,配置iSCSI。输入ls我们可以看到iSCSI初始状态,backstores下有四个节点:
block
fileio
pscsi
ramdis
实例需要使用block和fileio节点。创建block磁盘和fileio磁盘了。iscsi节点,需要创建服务端的配置和客户端的配置,以及客户端的认证。总体操作思路如下:
创建iSCSI磁盘
创建服务端程序配置
创建服务卷
创建客户端连接配置(启用CHAP认证)
重启target服务
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /> cd /backstores/block/ /> /backstores/block/ create idisk1 /dev/sdb created block storage idisk1 using /dev/sdb. /> ls o- / .................................................................................................................... [...] o- backstores ......................................................................................................... [...] | o- block ............................................................................................. [Storage Objects: 1] | | o- idisk1 ..................................................................... [/dev/sdb (10.0GiB) write-thru activated] | | o- alua .............................................................................................. [ALUA Groups: 1] | | o- default_tg_pt_gp .................................................................. [ALUA state: Active/optimized] /> cd /blockstores/fileio/ /> /backstores/fileio/ create idisk2 /opt/disk.img created idisk2 with 8388608. /> ls o- / .................................................................................................................... [...] o- backstores ......................................................................................................... [...] | o- block ............................................................................................. [Storage Objects: 1] | | o- idisk1 ..................................................................... [/dev/sdb (10.0GiB) write-thru activated] | | o- alua .............................................................................................. [ALUA Groups: 1] | | o- default_tg_pt_gp .................................................................. [ALUA state: Active/optimized] | o- fileio ............................................................................................ [Storage Objects: 1] | | o- idisk2 ................................................................. [/opt/disk.img (1.0GiB) write-back activated] | | o- alua .............................................................................................. [ALUA Groups: 1] | | o- default_tg_pt_gp .................................................................. [ALUA state: Active/optimized] /> cd /iscsi /> /iscsi/ create iqn.2025-01.ywmy.xyz:server Created target iqn.2025-01.ywmy.xyz:server. Created TPG 1. Global pref auto_add_default_protal=true Created default portal listening IPs (0.0.0.0), port 3260. /> ls o- / .................................................................................................................... [...] o- backstores ......................................................................................................... [...] | o- block ............................................................................................. [Storage Objects: 1] | | o- idisk1 ..................................................................... [/dev/sdb (10.0GiB) write-thru activated] | | o- alua .............................................................................................. [ALUA Groups: 1] | | o- default_tg_pt_gp .................................................................. [ALUA state: Active/optimized] | o- fileio ............................................................................................ [Storage Objects: 1] | | o- idisk2 ................................................................. [/opt/disk.img (1.0GiB) write-back activated] | | o- alua .............................................................................................. [ALUA Groups: 1] | | o- default_tg_pt_gp .................................................................. [ALUA state: Active/optimized] | o- pscsi ............................................................................................. [Storage Objects: 0] | o- ramdisk ........................................................................................... [Storage Objects: 0] o- iscsi ....................................................................................................... [Targets: 1] | o- iqn.2025-01.ywmy.xyz:server .................................................................................. [TPGs: 1] | o- tpg1 .......................................................................................... [no-gen-acls, no-auth] | o- acls ..................................................................................................... [ACLs: 0] | o- luns ..................................................................................................... [LUNs: 2] | o- portals ............................................................................................... [Portals: 0] | o- 0.0.0.0:3260 ................................................................................................ [OK] /> cd /iscsi/iqn.2025-01.ywmy.xyz:server/tgp1/luns> create /backstores/block/idisk1 Created LUN 0. /> cd /iscsi/iqn.2025-01.ywmy.xyz:server/tgp1/luns> create /backstores/fileio/idisk2 Created LUN 1. /> cd /iscsi/iqn.2025-01.ywmy.xyz:server/tpg1/acls /> /iscsi/iqn.2025-01.ywmy.xyz:server/tpg1/acls> create iqn.2025-01.ywmy.xyz:client Created node ACL for iqn.2025-01.ywmy.xyz:client Created mapped LUN 1. Created mapped LUN 0. /> cd /iscsi/iqn.2025-01.ywmy.xyz:server/tpg1/acls> cd iqn.2025-01.ywmy.xyz:client /> /iscsi/iqn.2025-01.ywmy.xyz:server/tg1/acls/iqn.2025-01.ywmy.xyz:client> set auth userid yourname /> /iscsi/iqn.2025-01.ywmy.xyz:server/tg1/acls/iqn.2025-01.ywmy.xyz:client> set auth password=your_password /> ls o- / .................................................................................................................... [...] o- backstores ......................................................................................................... [...] | o- block ............................................................................................. [Storage Objects: 1] | | o- idisk1 ..................................................................... [/dev/sdb (10.0GiB) write-thru activated] | | o- alua .............................................................................................. [ALUA Groups: 1] | | o- default_tg_pt_gp .................................................................. [ALUA state: Active/optimized] | o- fileio ............................................................................................ [Storage Objects: 1] | | o- idisk2 ................................................................. [/opt/disk.img (1.0GiB) write-back activated] | | o- alua .............................................................................................. [ALUA Groups: 1] | | o- default_tg_pt_gp .................................................................. [ALUA state: Active/optimized] | o- pscsi ............................................................................................. [Storage Objects: 0] | o- ramdisk ........................................................................................... [Storage Objects: 0] o- iscsi ....................................................................................................... [Targets: 1] | o- iqn.2025-01.ywmy.xyz:server .................................................................................. [TPGs: 1] | o- tpg1 .......................................................................................... [no-gen-acls, no-auth] | o- acls ..................................................................................................... [ACLs: 1] | | o- iqn.2025-01.ywmy.xyz:client ..................................................................... [Mapped LUNs: 2] | | o- mapped_lun0 ........................................................................... [lun0 block/idisk1 (rw)] | | o- mapped_lun1 .......................................................................... [lun1 fileio/idisk2 (rw)] | o- luns ..................................................................................................... [LUNs: 2] | | o- lun0 ................................................................ [block/idisk1 (/dev/sdb) (default_tg_pt_gp)] | | o- lun1 .......................................................... [fileio/idisk2 (/opt/disk.img) (default_tg_pt_gp)] | o- portals ............................................................................................... [Portals: 1] | o- 0.0.0.0:3260 ................................................................................................ [OK] o- loopback .................................................................................................... [Targets: 0] o- vhost ....................................................................................................... [Targets: 0] o- xen-pvscsi .................................................................................................. [Targets: 0] /> exit Global pref auto_save_on_exit=true Configuration saved to /etc/target/saveconfig.json
使用exit命令退出targetcli,配置文件自动保存。
客户端配置 客户端使用iscsi-initiator-utils。我们仍然使用openEuler环境作为客户端。
iscsi-initiator-utils 安装 通过yum直接从源里安装。
1 yum install iscsi-initiator-utils
在openEuler中安装iscsi-initiator-utils,安装完成后,启动iscsd服务,一直报错,无法启动服务,卸载之后,重新安装也报错。查看日志发现一些问题。
1 2 3 4 5 6 Jan 7 12:26:59 ha2 systemctl[8751]: [systemctl start iscsid] called by PID 8660 (-bash) Jan 7 12:26:59 ha2 systemd[1]: Starting Open-iSCSI... Jan 7 12:26:59 ha2 iscsid[8753]: /sbin/iscsid: error while loading shared libraries: /lib64/libisns.so.0: file too short Jan 7 12:26:59 ha2 systemd[1]: iscsid.service: Main process exited, code=exited, status=127/n/a Jan 7 12:26:59 ha2 systemd[1]: iscsid.service: Failed with result 'exit-code' . Jan 7 12:26:59 ha2 systemd[1]: Failed to start Open-iSCSI.
可以看到启动有依赖报错。iscsi-initiator-utils,这个程序包在ubuntu下,直接安装为open-iscsi。libisns.so.0,我猜想也是依赖问题,找到程序目录,发下文件为空。所以,尝试单独安装依赖。
1 2 3 yum install open-isns yum install open-iscsi yum install iscsi-initiator-utils
配置客户端程序 安装iscsi-initiator-utils程序后,在/etc/iscsi/目录下生成配置文件。我们需要编辑initiatorname.iscsi。写入服务器配置的连接器名称。
1 2 3 vim initiatorname.iscsi InitiatorName=iqn.2025-01.ywmy.xyz:client
配置认证,我们需要配置iscsid.conf文件。
1 2 3 4 node.startup = manual node.session.auth.username = username node.session.auth.password = password
启动客户端程序 客户端查找服务端程序:
1 2 iscsiadm -m discovery -t st -p 192.168.31.150 192.168.31.150:3260,1 iqn.2025-01.ywmy.xyz:server
客户端连接iSCSI服务端:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 systemctl status iscsid ● iscsid.service - Open-iSCSI Loaded: loaded (/usr/lib/systemd/system/iscsid.service; disabled; preset: disabled) Active: active (running) since Tue 2025-01-07 13:00:11 CST; 4s ago TriggeredBy: × iscsid.socket Docs: man:iscsid(8) man:iscsiadm(8) Main PID: 10769 (iscsid) Status: "Ready to process requests" Tasks: 1 (limit : 9104) Memory: 1.8M () CGroup: /system.slice/iscsid.service └─10769 /sbin/iscsid -f 1月 07 13:00:11 ha2 systemd[1]: Starting Open-iSCSI... 1月 07 13:00:11 ha2 systemd[1]: Started Open-iSCSI.
启动服务后,连接iSCSI服务。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 iscsiadm -m node -T iqn.2025-01.ywmy.xyz:server -p 192.168.31.150 --login Logging in to [iface: default, target: iqn.2025-01.ywmy.xyz:server, portal: 192.168.31.150,3260] Login to [iface: default, target: iqn.2025-01.ywmy.xyz:server, portal: 192.168.31.150,3260] successful. lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 20G 0 disk ├─sda1 8:1 0 1M 0 part ├─sda2 8:2 0 1G 0 part /boot └─sda3 8:3 0 19G 0 part ├─openeuler-root 253:0 0 17G 0 lvm / └─openeuler-swap 253:1 0 2G 0 lvm [SWAP] sdb 8:16 0 10G 0 disk sdc 8:32 0 1G 0 disk
我们已经看到了远程挂载的磁盘,sdb和sdc已经有了。linux端的配置和验证结束。