紆余曲折ありましたが、バックプレーンボードが届いたので、取り付けて1TB✕14でRAIDを組もうと思います。
IBM x3650 M4にはIBM m5110eというRAIDカードが搭載されてますが、基本的にはRAID6には対応してません。また、ディスクをRAID構成なしで放置するとエラーを吐くので、RAID0で１台構成にする必要があります。
数台程度なら良いのですが、14台をWebBIOSで設定するのは骨が折れたので、コマンドからなんとかしようと思います。
MegaCliをインストールして、エイリアスを切っておきます。以降エイリアスありでコマンドは記載します。
wget https://ginernet.com/downloads/MegaCli-8.07.14-1.noarch.rpm rpm -Uhv MegaCli-8.07.14-1.noarch.rpm alias megacli='/opt/MegaRAID/MegaCli/MegaCli64'
とりあえず、カードの情報を取ります。
増設の都合でRAIDカードが２枚ありますが、バックプレーンにはSASExpanderが付属だったので、Controller 1の方は今回使いません。
# megacli -AdpGetPciInfo -aAll PCI information for Controller 0 -------------------------------- Bus Number : 16 Device Number : 0 Function Number : 0 PCI information for Controller 1 -------------------------------- Bus Number : 8b Device Number : 0 Function Number : 0 Exit Code: 0x00
物理ディスク構成を確認します。
長いので中略ですが、boot用にSAS 137GB✕2とSATA 1TB✕14です。
# megacli -PDList -aALL Adapter #0 Enclosure Device ID: 20 Slot Number: 0 Drive's position: DiskGroup: 0, Span: 0, Arm: 0 Enclosure position: 1 Device Id: 19 WWN: 5000C50068629614 Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SAS Raw Size: 136.731 GB [0x11176d60 Sectors] Non Coerced Size: 136.231 GB [0x11076d60 Sectors] Coerced Size: 135.972 GB [0x10ff2000 Sectors] Sector Size: 512 Logical Sector Size: 512 Physical Sector Size: 512 Firmware state: Online, Spun Up Commissioned Spare : No Emergency Spare : No Device Firmware Level: B63B Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x5000c50068629615 SAS Address(1): 0x0 Connected Port Number: 0(path0) Inquiry Data: IBM-ESXSST9146853SS B63B6XM2PYVF1018B63B IBM FRU/CRU: 90Y8927 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 6.0Gb/s Link Speed: 6.0Gb/s Media Type: Hard Disk Device Drive: Not Certified Drive Temperature :28C (82.40 F) PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 6.0Gb/s Port-1 : Port status: Active Port's Linkspeed: 6.0Gb/s Drive has flagged a S.M.A.R.T alert : No Enclosure Device ID: 20 Slot Number: 1 Drive's position: DiskGroup: 0, Span: 0, Arm: 1 Enclosure position: 1 Device Id: 18 WWN: 5000C500683F6544 Sequence Number: 2 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SAS Raw Size: 136.731 GB [0x11176d60 Sectors] Non Coerced Size: 136.231 GB [0x11076d60 Sectors] Coerced Size: 135.972 GB [0x10ff2000 Sectors] Sector Size: 512 Logical Sector Size: 512 Physical Sector Size: 512 Firmware state: Online, Spun Up Commissioned Spare : No Emergency Spare : No Device Firmware Level: B63B Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x5000c500683f6545 SAS Address(1): 0x0 Connected Port Number: 0(path0) Inquiry Data: IBM-ESXSST9146853SS B63B6XM2P96W1018B63B IBM FRU/CRU: 90Y8927 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 6.0Gb/s Link Speed: 6.0Gb/s Media Type: Hard Disk Device Drive: Not Certified Drive Temperature :28C (82.40 F) PI Eligibility: No Drive is formatted for PI information: No PI: No PI Port-0 : Port status: Active Port's Linkspeed: 6.0Gb/s Port-1 : Port status: Active Port's Linkspeed: 6.0Gb/s Drive has flagged a S.M.A.R.T alert : No Enclosure Device ID: 20 Slot Number: 2 Enclosure position: 1 Device Id: 26 WWN: 5000c5003539094f Sequence Number: 1 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 931.512 GB [0x74706db0 Sectors] Non Coerced Size: 931.012 GB [0x74606db0 Sectors] Coerced Size: 930.390 GB [0x744c8000 Sectors] Sector Size: 512 Logical Sector Size: 512 Physical Sector Size: 512 Firmware state: Unconfigured(good), Spun Up Device Firmware Level: SN02 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x500507603ea9640b Connected Port Number: 0(path0) Inquiry Data: 9XG0ERFCST91000640NS SN02 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 6.0Gb/s Link Speed: 6.0Gb/s Media Type: Hard Disk Device Drive: Not Certified Drive Temperature :27C (80.60 F) PI Eligibility: No Drive is formatted for PI information: No PI: No PI Drive's NCQ setting : Enabled Port-0 : Port status: Active Port's Linkspeed: 6.0Gb/s Drive has flagged a S.M.A.R.T alert : No ### 長いので中略 #### Enclosure Device ID: 20 Slot Number: 15 Enclosure position: 1 Device Id: 35 WWN: 5000c50035121495 Sequence Number: 1 Media Error Count: 0 Other Error Count: 0 Predictive Failure Count: 0 Last Predictive Failure Event Seq Number: 0 PD Type: SATA Raw Size: 931.512 GB [0x74706db0 Sectors] Non Coerced Size: 931.012 GB [0x74606db0 Sectors] Coerced Size: 930.390 GB [0x744c8000 Sectors] Sector Size: 512 Logical Sector Size: 512 Physical Sector Size: 512 Firmware state: Unconfigured(good), Spun Up Device Firmware Level: SN02 Shield Counter: 0 Successful diagnostics completion on : N/A SAS Address(0): 0x500507603ea96408 Connected Port Number: 0(path0) Inquiry Data: 9XG0BV54ST91000640NS SN02 FDE Capable: Not Capable FDE Enable: Disable Secured: Unsecured Locked: Unlocked Needs EKM Attention: No Foreign State: None Device Speed: 6.0Gb/s Link Speed: 6.0Gb/s Media Type: Hard Disk Device Drive: Not Certified Drive Temperature :26C (78.80 F) PI Eligibility: No Drive is formatted for PI information: No PI: No PI Drive's NCQ setting : Enabled Port-0 : Port status: Active Port's Linkspeed: 6.0Gb/s Drive has flagged a S.M.A.R.T alert : No Adapter #1 Exit Code: 0x00
RAID0でEnclosure Device ID: 20,Slot Number: 2のディスクでRAIDを組みます。
-r0がRAIDレベル、[20:2]がエンクロージャー:スロット、-a0がアダプタ0の指定です。
# megacli -CfgLdAdd -r0 [20:2] -a0 Adapter 0: Created VD 1 Adapter 0: Configured the Adapter!! Exit Code: 0x00
かんたんですね。
残りも以下のコマンドで同様に作ります。
megacli -CfgLdAdd -r0 [20:3] -a0 megacli -CfgLdAdd -r0 [20:4] -a0 megacli -CfgLdAdd -r0 [20:5] -a0 megacli -CfgLdAdd -r0 [20:6] -a0 megacli -CfgLdAdd -r0 [20:7] -a0 megacli -CfgLdAdd -r0 [20:8] -a0 megacli -CfgLdAdd -r0 [20:9] -a0 megacli -CfgLdAdd -r0 [20:10] -a0 megacli -CfgLdAdd -r0 [20:11] -a0 megacli -CfgLdAdd -r0 [20:12] -a0 megacli -CfgLdAdd -r0 [20:13] -a0 megacli -CfgLdAdd -r0 [20:14] -a0 megacli -CfgLdAdd -r0 [20:15] -a0
IMM2から確認するとRAID0で14プールできていることがわかります。
それぞれパーティションを切ります。
#parted -s -a optimal /dev/sdb -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdc -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdd -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sde -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdf -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdg -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdh -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdi -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdj -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdk -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdl -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdm -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdn -- mkpart primary xfs 1 -1 #parted -s -a optimal /dev/sdo -- mkpart primary xfs 1 -1
前回同様にmdでRAID6を組みます。
# mdadm --create /dev/md/md12tb --level=raid6 --name=md12tb --raid-devices=14 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 # mdadm --detail --scan > /etc/mdadm.conf
ベンチします。
前回が432MB/sだったので、3倍弱ですね。
# hdparm -t /dev/md/md12tb /dev/md/md12tb: Timing buffered disk reads: 3620 MB in 3.00 seconds = 1206.25 MB/sec