Ubuntu 10.04 LTS RAID1 Home Server setup with USB stick support
The project was canceled because it does not work reliable over time (month).
The job was to convert the hard disk setup of an already running home server (Eee PC 900A, USB external 2.5" disk 250 GB) into a RAID1 array together with a second external 250 GB hard disk in order to improve the uptime of the server. After loosing a new Samsung 1TB hard disk in a PC after 3 weeks use I thought this is the weakest spot of the server.
The GRUB2 boot loader should be configured in such a way that the system will still be able to boot if one of the hard drives fails (no matter which one). Also the loss of one hard disk should be reported via email.
The email reporting works in a test.
Normally it is recommended to attach a second, same size, hard disk to the computer and copy force and back, see link. The disadvantage of this method is, that you have some downtime of the server, especially in the case of any handling failure.
So I first made a copy of the live disk, then tried with a third disk (same size = 250 GB) on a second off-line computer to create the RAID1 array.
Because of some boot failures when creating the RAID1 array, finally I used an Ubuntu 10.04.1 desktop edition USB stick, 4 GB, to make the setup with two new disks easier (USB, 250 GB, from Maxtor and Western Digital, named later MX250 and WD250).
After trying the server edition of Ubuntu with little success (I am a GUI lover), I switched to the desktop edition.
What took me a lot of time was the Linux kernel selection. I was not aware that the desktop kernel (-generic) did not have a RAID support. The standard kernel of the Ubuntu server edition does have it, -generic-pae (Physical Address Extension). Later I found out, that this kernel with included RAID support is also available in the desktop edition, you just have to install it. Unfortunately the RAID support is not mentioned at all in the notes, you have to figure that out by yourself (trail and error). An alternative is the Alternate edition, which uses a terminal setup, but allows RAID and LVM setup.
Conclusion
At 2011-01-23 I gave up with the RAID1 array, because I found out, that Ubuntu is using an old Version (2.6.7.1, even in Ubuntu 10.10) of mdadm, while Debian is using version 3.2.1 (March 2010), which should work more stable. The use of the bitmap feature improved on one hand the resynchronize speed drastically, but on the other hand distorted the Ubuntu installation, e.g. after a resynchronize the folder /etc was missing. Then I had difficulties to install the GRUB2 boot loader, because the RAID device name /dev/md0 changed after a boot to /dev/md_d0. That could be fixed, but I expected with an RAID1 array more stability, not less. Version 3.2.1 of mdadm is expected to be integrated in the Ubuntu release 11.04 (Natty Narwhal) https://bugs.launchpad.net/bugs/368986. If a back port will appear in Ubuntu 10.04 I might give it another try.
The actual version of mdadm is 3.1.4 (Aug. 2010), see http://www.kernel.org/pub/linux/utils/raid/mdadm/ANNOUNCE, but it looks like it has a boot problem with GRUB2, because of the changed meta data version 1.2, http://www.debian.org/releases/squeeze/hppa/release-notes/ch-information.de.html#mdadm-metadata.
Update at 2011-01-31, RAID1 is working
After finding out with the shell script boot_info_script055.sh that the GRUB2 boot loader must be installed on partition #1 only, I gave the RAID1 array another try, which was then successful, see UbuntuRaid1Log.
Hard Disk Layout
The old server disk has the following partitions:
/dev/sda1, ext4, mount point / /dev/sda2, swap, 2 GB
The target server RAID 1 configuration should be:
/dev/md0 (made up of /dev/sda1 and /dev/sdb1): mount point /boot, ext2 /dev/md1 (made up of /dev/sda2 and /dev/sdb2): swap /dev/md2 (made up of /dev/sda3 and /dev/sdb3): mount point /, ext3 or ext4
USB Stick problem, Ubuntu 10.04.1
There is a bug in the USB stick generation, described in https://bugs.launchpad.net/ubuntu/+source/syslinux/+bug/617779.
If you boot with either the "Desktop" edition or the "Server" edition, while booting up you get the error message: "Unknown keyword in configuration file: gfxboot ...". Fortunately the workaround is simple, just type "help" at the "boot:" prompt and ENTER. Then the normal boot procedure is running.
Prerequisites
- A server computer with one disk /dev/sda.
- A second computer, preferably the same type as the server computer, with cable network connection.
Two hard disks >= the size of the server hard disk, I use USB external 2.5" disks, because of low power consumption.
One USB stick >= 1 GB size for holding the Ubuntu 10.04.1 desktop edition install version (xxx.iso).
One USB stick >= 4 GB size for holding the Ubuntu 10.04.1 installed desktop edition.
To work with the Ubuntu live version only is very time consuming if you have a lot of reboots.
Therefore it is more convenient to use an installed Ubuntu version.Partition the hard disks with Gparted in 3 partitions.
Because of track length rounding of Gparted it is wise to keep a few megabytes between the partitions free, especially if you have different disk brands.
It is also useful to keep one megabyte free at the beginning of the disk, and some megabytes at the end for adjusting the partitions to the same size, and having free space for the RAID superblock, which could be at the beginning or the end of the disk, depending of the version.
It may not all be necessary, but all my problems went away when I did so. Troubleshooting can be really tiring in that field. It looks like a multiple choice game.Partition 1, 500 MB size, primary, ext2, mount point /boot, /dev/sdb1 or /dev/sdc1, /dev/md0 Partition 2, 2 GB size, primary, swap, dev/sdb2 or /dev/sdc2, dev/md1 Partition 3, rest of the disk, primary, mount point /, /dev/sdb3 or /dev/sdc3, /dev/md2
Set also the boot flag in the boot partition.
Check all file systems for errors:$ sudo fsck.ext2 /dev/sdb1 $ sudo fsck.ext2 /dev/sdc1 $ sudo fsck.ext3 /dev/sdb3 $ sudo fsck.ext3 /dev/sdc3
If sudo fsck.ext2 /dev/sdb1 is complaining about different file system/partition size, do a resize2fs to that partition. The assembly of a RAID array will work only if fsck has no more complains.
Do not set the RAID flags in the partition in this state, because a running mdadm daemon will grab it, an it is more difficult to make file system maintenance.
If you have already set the RAID flag and want to do a sudo fsck.ext2 /dev/sdb1 for example, then you must stop the mdadm daemon access before:$ sudo mdadm --stop /dev/md0 (or what is your RAID naming for /dev/sdb1) $ sudo fsck.ext2 /dev/sdb1
Copy server data
Now connect the second hard disk to the server and copy all the data.
In order to avoid problems with any running services, stop them first (for example):
$ sudo /etc/init.d/mysql stop $ sudo /etc/init.d/samba stop $ sudo /etc/init.d/networking stop
Next mount the second disk:
$ sudo mount /mnt /dev/sdb3
For the copying you have to consider to not copy dynamic folders, /var/run and /var/lock should not give a problem:
$ sudo cp -dpRvx $(ls --color=never -I boot -I dev -I proc -I sys /) /mnt legend for dir options: -d keep links -p keep date/time stamps -R copy sub folders recursive -v show all files for verification -x do not copy mounted file systems (the second disk) last create the missing folders: $ sudo mkdir /mnt/dev $ sudo mkdir /mnt/proc $ sudo mkdir /mnt/sys
After the (long) copy process unmount (and unplug) the second disk and reboot:
$ sudo umount /mnt $ sudo reboot unplug second disk
RAID 1 Installation with the support of an USB stick
Also have a look at https://help.ubuntu.com/community/Installation/FromUSBStick. Unfortunately it describes Ubuntu version 09.10 only, but was updated on Dez. 2010. It will hopefully improve your learning curve.
In order to get not confused with the internal SSD of the netbook (/dev/sdx numbering), I removed it for that action.
In order to make the RAID 1 installation easier, and got tired to always setup the Ubuntu live USB stick, I decided to create an Ubuntu 10.04.1 desktop installation on an 4 GB USB stick with RAID support. Because it is not so easy I will describe in short the necessary steps:
Use an actual >= 10.04.1 Ubuntu desktop install .iso file, to create a start media USB stick (>= 1 GB) to boot your computer.
Boot with that start media, in order to have the suitable Ubuntu version.
The Ubuntu installation on another USB stick itself will need about 2.7 GB space.
Plug in a second USB stick >= 4 GB and use Gparted to partition and format it:
Partition 1 type primary with file system ext4 leaving 256 MB free at the end.
Partition 2 type extended and partition 5 type logical as swap.Boot with the 4 GB USB stick, and get a network connection.
Install your language pack, if not using English, and install mdadm for the RAID setup.$ sudo apt-get install mdadm while installing you will be asked for the postfix setup. If the OK button is not highlighted in red, hit the TAB key, and acknowledge all other questions with ENTER. Show the version number of the installed package: $ sudo apt-cache --installed showpkg mdadm Package: mdadm Versions: 2.6.7.1-1ubuntu15 (/var/lib/apt/lists/de.archive.ubuntu.com_ubuntu_dists_lucid_main_binary-i386_Packages) (/var/lib/dpkg/status)
After that, run the update job (about 200 MB, > 2 h installation time, 2010-12-24)
Install linux-image-2.6.32-27-generic-pae, which includes RAID support!
- Shut the computer down, remove the USB stick and plug it in your target computer, or just reboot. In order to see which kernel version is installed type:
$ uname -a Linux <user name> 2.6.32.27-generic-pae #49-Ubuntu SMP Thu Dec 2 00:07:52 2010 i686 GNU/Linux
Install some support programs with synaptic: gparted, htop, mc
Now plug in the two already partitioned and formatted hard disks.
To make sure that there are no remains from previous RAID installations we run the following commands:$ sudo mdadm --zero-superblock /dev/sdb1 $ sudo mdadm --zero-superblock /dev/sdb2 $ sudo mdadm --zero-superblock /dev/sdb3 $ sudo mdadm --zero-superblock /dev/sdc1 $ sudo mdadm --zero-superblock /dev/sdc2 $ sudo mdadm --zero-superblock /dev/sdc3
If there are no remains from previous RAID installations, each of the above commands will throw an error like this one (which is nothing to worry about):mdadm: Unrecognised md component device - /dev/sdb1
- Now its time to create our RAID1 arrays:
$ sudo mdadm --create /dev/md0 --level=1 --raid-disks=2 /dev/sdb1 /dev/sdc1 $ sudo mdadm --create /dev/md1 --level=1 --raid-disks=2 /dev/sdb2 /dev/sdc2 $ sudo mdadm --create /dev/md2 --level=1 --raid-disks=2 /dev/sdb3 /dev/sdc3
Modify the GRUB2 control file /etc/default/grub.
Remove the kernel parameters quiet and splash to allow kernel messages on the screen
Set bootdegraded=yes in order to allow a single hard disk from the RAID1 array to boot.#GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" GRUB_CMDLINE_LINUX_DEFAULT="bootdegraded=yes"
Write a new GRUB2 boot loader to disk:First check if "mdamd" is activ: $ cat /proc/mdstat xxx If "mdadm" does not run, start it with: $ sudo /etc/init.d/mdadm start If /media/boot does not exist: $ sudo mkdir /media/boot $ sudo mount -t ext3 /dev/md2 /mnt $ sudo mount --bind /dev /mnt/dev $ sudo mount --bind /proc /mnt/proc $ sudo mount --bind /sys /mnt/sys $ sudo mount -t ext2 /dev/md0 /media/boot $ sudo mount --bind /media/boot /mnt/boot # cover original /boot $ sudo chroot /mnt $ grub-install /dev/sdb $ grub-install /dev/sdc $ grub-install --recheck /dev/sdb # just to be sure $ grub-install --recheck /dev/sdc # and update our GRUB2 bootloader configuration: $ update-grub $ update-initramfs -u -k all
The file /boot/grub/grub.cfg should have the following entry.
It is important, that the -generic-pae kernel is listed, and that the root device is (md0).### BEGIN /etc/grub.d/10_linux ### menuentry 'Ubuntu, mit Linux 2.6.32-27-generic-pae' --class ubuntu --class gnu-linux --class gnu --class os { recordfail insmod raid insmod mdraid insmod ext2 set root='(md0)' search --no-floppy --fs-uuid --set 135c7575-a651-4c6f-8a4d-630f97b6691b linux /vmlinuz-2.6.32-27-generic-pae root=UUID=624cbccb-5008-40aa-94a1-35d23000b776 ro bootdegraded=yes initrd /initrd.img-2.6.32-27-generic-pae }
Last give up the chroot environment:Ctrl+D (to exit out of chroot) $ sudo umount /mnt/dev $ sudo umount /mnt
- Power off the computer, pull the USB stick, power on the computer and see if it works.
$ sudo poweroff
Part 2, Using Ubuntu Desktop in VirtualBox 3.2.12 for RAID1
This is for Tests/Training only. The benefit is that you can use smaller size virtual disks (about 4 GB each), and can make easy copies of it.
With the help from http://www.ralf-schmidt.de/Computer/Howto_RAID1-LVM_Grub.html
Unfortunately in the process I got 2 computer freezes, so I gave up this method.
Virtualbox Additions problem with -pae kernel
When you install the Virtualbox Additions, the compiling of the -pae kernel modules will fail. That has for example the drawback that the Host/Guest clipboard does not work. The files /home/<user>/.vboxclient-clipboard.pid, vboxclient-display.pid and vboxclient-seamless.pid are missing. Have a look also at /var/log/vboxadd-install.log.
The problem was caused from not installed -pae Linux headers.
Search in Synaptic for Linux headers. If there is a green mark at the normal -generic linux-headers-2.6.32-xx and not at the -generic-pae version, then mark it for installation. A second case is at linux-headers-generic and linux-headers-generic-pae. After the installation of the -pae headers the compilation of the vbox kernel modules will work.
Hard Disk Setup
First I started with 2 virtual disks, then I added a third disk because it appears easier to use 3 disks to create a RAID1 array:
Disk 1 /dev/sda - Ubuntu 10.04.1 Desktop installation with -pae boot kernel with RAID support The installation is derived from a 4 GB USB stick, which was setup on another computer. Disk 2 /dev/sdb - Ubuntu 10.04.1 Desktop installation, normal kernel for boot Disk 3 /dev/sdc - partitioned the same as Disk2, otherwise empty
The benefit of this configuration is, that I always have a stable bootable installation and can manipulate the two other disks. After the first stable setup, I copy the virtual hard disks (.vdi files), so I have a fall back, just to save time in case of any failure.
The device sequence /dev/sdX in virtualbox can be changed by removing the .vdi files in the disk manager and loading them afterwards in the wished sequence.
The boot priority can be changed by pressing key F12 when the VirtualBox mask shows up after start.
In order to simulate the situation with USB stick boot on the netbook computer you have to do the following:
Copy the working USB stick with Ubuntu 10.04.1 with -pae kernel to an .iso file.
Version Mac OS X: Use the hard disk manager to create an USB stick image. In Image Format select: "DVD/CD-Master" Rename the resultant file from .cdr to Ubuntu100441usb.iso. Version Win32: Use Program USB Image Tool.exe from Alexander Beug, www.alexpage.de (Freeware)
Next convert (wrap) the .iso file to a .vdi file (Virtual Disk)In Mac OS X open a terminal window $ cd to the folder with the .iso file $ /Applications/VirtualBox.app/Contents/MacOS/VBoxManage convertfromraw Ubuntu10041usb.iso Ubuntu10041usb.vdi In Win32 use VBoxManage.exe and the command line, e.g. "c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" convertfromraw Ubuntu10041usb.iso Ubuntu10041usb.vdi
In the first trial I used just 2 disks only, /dev/sda and /dev/sdb. Later I added a third disk for easier handling and changed the disk order, like I described at the beginning.
As an alternative install Ubuntu version 10.04.1 LTS in a 4 GB virtual disk under VirtualBox version 3.2.12. The desktop edition will give easier handling with a mixture from command line and GUI.
- Partition 1 - 512 MB, primary, ext2, /boot
- Partition 2 - 256 MB, primary, swap
- Partition 3 - 3400 MB, primary, ext3, /
Mark the first partition with the boot and raid flag, the second and third partition with the raid flag.
- Install the RAID support and other programs:
$ sudo apt-get install mdadm gparted mc htop testdisk At the postfix installation, included with mdadm, just hit always ENTER if you do not use it. open an admin file browser: $ sudo gksu nautilus edit file /etc/group and append to "disk:x:6:" your user name logout - login to apply the group attributes reboot Start Ubuntu update (203 MB, 2010-12-26) Install VBOXADDITIONS for Linux with a double click on autorun.sh
- Next create a second virtual harddisk of 4.05 GB size, like the first one plus an extra. Check the device names with:
$ sudo fdisk -l # should be /dev/sda and /dev/sdb Partition and format the second hard disk with Gparted: first define a "msdos" partition table. in order to get the same values, click OFF "adjust to cylinder size". start the job, after that mark the first partition with the "boot" and "raid" flag, the second and third with the "raid" flag.
Convert the new disk2 /dev/sdb to a RAID array /dev/md0, /dev/md1 and /dev/md2. Just the new partitions of disk2 are used. Instead of the partitions of disk1 /dev/sda type missing. The partitions of disk1 will be added later. Take care to not using the data of disk1.
$ sudo mdadm --create /dev/md0 --level=1 --raid-disk=2 missing /dev/sdb1 # /boot mdadm: /dev/sdb1 appears to contain an ext2fs file system size=498688K mtime=Thu Jan 1 01:00:00 1970 Continue creating array? y mdadm: array /dev/md0 started. $ sudo mdadm --create /dev/md1 --level=1 --raid-disk=2 missing /dev/sdb2 # swap mdadm: Cannot open /dev/sdb2: Device or resource busy mdadm: create aborted Solution: start Gparted and unmount /dev/sdb2, then try again mdadm: array /dev/md1 started. $ sudo mdadm --create /dev/md2 --level=1 --raid-disk=2 missing /dev/sdb3 # root Answer to the following question with yes: mdadm: /dev/sdb3 appears to contain an ext2fs file system size=3819520K mtime=Thu Jan 1 01:00:00 1970 Continue creating array? y mdadm: array /dev/md2 started.
Check for the state of the RAID array the virtual file /proc/mdstat. All show [_U], "_" means missing, "U" means Up.
$ cat /proc/mdstat Personalities : [raid1] md1 : active raid1 sdb2[1] 249792 blocks [2/1] [_U] md2 : active raid1 sdb3[1] 3319744 blocks [2/1] [_U] md0 : active raid1 sdb1[1] 498624 blocks [2/1] [_U] unused devices: <none>
In order to not loose the RAID after boot, append them to the file /etc/mdadm/mdadm.conf.
First start an admin file browser: $ gksu nautilus and give the file read-write permission for all $ sudo mdadm --examine --scan >> /etc/mdadm/mdadm.conf then give the file read-only permission for all
If the file does contain older RAID definitions, please delete them first.... # This file was auto-generated on Sun, 26 Dec 2010 09:15:20 +0100 # by mkconf $Id$ ARRAY /dev/md0 level=raid1 num-devices=2 UUID=1e95604a:a59ae7c8:753e14fc:0999d422 ARRAY /dev/md1 level=raid1 num-devices=2 UUID=1cdde281:f64347f4:753e14fc:0999d422 ARRAY /dev/md2 level=raid1 num-devices=2 UUID=80ec79d4:f0722fce:753e14fc:0999d422
Because mdadm.conf will be used in initramfs you have also to update initrd:
$ sudo update-initramfs -u -k all
From the next boot on, Linux should use the new RAID1 configuration. Therefore you have to modify the file /etc/fstab. First make a backup copy of this file.
$ sudo cp /etc/fstab /etc/fstab.old show the UUID's, attention: blkid /dev/md0 will NOT show the right value! $ sudo mdadm --examine --scan copy the UUID's in /etc/fstab via admin file browser nautilus and gedit: ... # / was on /dev/sda3 during installation UUID=80ec79d4:f0722fce:753e14fc:0999d422 / ext4 errors=remount-ro 0 1 # / was on /dev/sda1 during installation UUID=1e95604a:a59ae7c8:753e14fc:0999d422 /boot ext2 defaults 0 2 # swap was on /dev/sda2 during installation UUID=1cdde281:f64347f4:753e14fc:0999d422 none swap sw 0 0
Copy data from your Ubuntu installation to the new partition, disk2. In order to copy the partition, you have to mount them first:
$ sudo mount -t ext3 /dev/md2 /mnt mount: wrong fs type, bad option, bad superblock on /dev/md2, missing codepage or helper program, or other error $ dmesg | tail ... EXT3-fs (md2): bad geometry: block count 829952 exceeds size of device (829936 blocks) https://bugs.launchpad.net/ubuntu/+source/partman-base/+bug/569900 - leave always at the end of the disk 1 MB free - start desktop edition, Gparted and shorten /dev/sda3 and /dev/sdb3 - try bootoption raid=noautodetect to get the kernel detection out of the way
So I shorted the partitions /dev/sdb3 by 64 MB with Gparted. In order to avoid problems if you use 2 disks only, please stop all daemons, including network.as an example: $ sudo /etc/init.d/mysql stop $ sudo /etc/init.d/samba stop $ sudo /etc/init.d/nsf-kernel-server stop Network: right click on the network icon, then deactivate by a click on "network activate"
Now, using 3 disks, this is fortunately no longer necessary.Next step in the 3 disk setup is to set in all partitions in disk 2 the raid flag with Gparted, and reboot. In the next step we will see, if our efforts where sucessful, when we create the arrays:
The swap partition: $ sudo mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdb2 /dev/sdc2 mdadm: /dev/sdc2 appears to be part of a raid array: level=raid1 devices=2 ctime=Sun Dec 26 18:59:38 2010 Continue creating array? y mdadm: array /dev/md1 started. $ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid1 sdc2[1] sdb2[0] 249792 blocks [2/2] [UU] [============>........] resync = 63.3% (158208/249792) finish=0.0min speed=15820K/sec The boot partition. $ sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1 [sudo] password for rudi: mdadm: /dev/sdb1 appears to contain an ext2fs file system size=498688K mtime=Tue Jan 11 19:41:53 2011 mdadm: /dev/sdc1 appears to contain an ext2fs file system size=498688K mtime=Thu Jan 1 01:00:00 1970 mdadm: /dev/sdc1 appears to be part of a raid array: level=raid1 devices=2 ctime=Sun Dec 26 18:56:44 2010 Continue creating array? y mdadm: array /dev/md0 started. $ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md0 : active raid1 sdc1[1] sdb1[0] 498624 blocks [2/2] [UU] md1 : active raid1 sdc2[1] sdb2[0] 249792 blocks [2/2] [UU] Last the root partition: $ sudo mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sdb3 /dev/sdc3 In order to see the progress of the sync process type: $ watch -n 2 cat /proc/mdstat Finally it looks like: $ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md2 : active raid1 sdc3[1] sdb3[0] 3306752 blocks [2/2] [UU] md0 : active raid1 sdc1[1] sdb1[0] 498624 blocks [2/2] [UU] md1 : active raid1 sdc2[1] sdb2[0] 249792 blocks [2/2] [UU] unused devices: <none> Now lets have a look at the UUID's: rudi@rudi-usb:~$ sudo mdadm --examine --scan ARRAY /dev/md0 level=raid1 num-devices=2 UUID=9ed26cd8:98d1c021:a78ec32f:6eb6df8d ARRAY /dev/md1 level=raid1 num-devices=2 UUID=1d0a5e9a:61162487:a78ec32f:6eb6df8d ARRAY /dev/md2 level=raid1 num-devices=2 UUID=f089bdc5:f728badf:a78ec32f:6eb6df8d
If in case after a reboot one of the array is not complete, you can assemble them like:
# First check the status: rudi@rudi-usb:~$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md_d2 : inactive sdc3[0](S) 3306752 blocks md_d0 : inactive sdb1[1](S) 498624 blocks md_d1 : active raid1 sdc2[0] sdb2[1] 249792 blocks [2/2] [UU] rudi@rudi-usb:~$ sudo mdadm --assemble /dev/md_d0 /dev/sdb1 /dev/sdc1 mdadm: /dev/md_d0 has been started with 2 drives. rudi@rudi-usb:~$ sudo mdadm --assemble /dev/md_d2 /dev/sdb3 /dev/sdc3 mdadm: /dev/md_d2 has been started with 2 drives. # now check if the arrays are OK - it it done rudi@rudi-usb:~$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md_d2 : active raid1 sdc3[0] sdb3[1] 3306752 blocks [2/2] [UU] md_d0 : active raid1 sdc1[0] sdb1[1] 498624 blocks [2/2] [UU] md_d1 : active raid1 sdc2[0] sdb2[1] 249792 blocks [2/2] [UU] # Unfortunately there is some confusion about the array naming: rudi@rudi-usb:~$ sudo mdadm -Es ARRAY /dev/md0 level=raid1 num-devices=2 UUID=263f7df8:1277e163:a78ec32f:6eb6df8d ARRAY /dev/md1 level=raid1 num-devices=2 UUID=9c24dfde:e419a23c:a78ec32f:6eb6df8d ARRAY /dev/md2 level=raid1 num-devices=2 UUID=4ad255e6:ec0071cb:a78ec32f:6eb6df8d rudi@rudi-usb:~$ blkid /dev/sda1: UUID="0b201bbc-110f-40ca-ae55-7808bc253fd0" TYPE="ext4" /dev/sda5: UUID="53d5a553-3588-4338-afa1-7430d84f57fd" TYPE="swap" /dev/sdb1: UUID="263f7df8-1277-e163-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sdb2: UUID="9c24dfde-e419-a23c-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sdb3: UUID="4ad255e6-ec00-71cb-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sdc1: UUID="263f7df8-1277-e163-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sdc2: UUID="9c24dfde-e419-a23c-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sdc3: UUID="4ad255e6-ec00-71cb-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/md_d0: UUID="85e75d2f-c675-4560-8b99-4f5c8249ed23" TYPE="ext2" /dev/md_d1: UUID="07921f70-7ad5-4d93-81af-e5769ae9e9c2" TYPE="swap" /dev/md_d2: UUID="45015e6d-4265-4ab0-8399-20da850c3763" TYPE="ext4" # The RAID array has a different name (/dev/md0) than the file system (/dev/md_d0)
Part 3, support informations
Make a disk bootable via chroot setup
chroot means, that you mount a second, non bootable, disk, and make it bootable with the help of the first disk.
First with gparted I format MX250 /dev/sdc with a 2 GB swap partition /dev/sdc1, then an EXT4 partition /dev/sdc2 -64 MB for the system. The -64 MB was for compensating differences of the two disks. On /dev/sdc1 and /dev/sdc2 the raid flag must be set and additional on /dev/sdc2 the boot flag. Then I shutdown the server, connected the server disk to the setup computer, boot with a live medium and copied the disk content. After that the server disk was unmounted, plugged again on the server and the server computer restarted. Now I have time to setup the RAID1 array.
This is a list of the commands executed in order,
- sudo fdisk -l
- sudo mount -t ext3 /dev/sdx2 /mnt
- sudo mount --bind /dev /mnt/dev
- sudo mount --bind /proc /mnt/proc
- sudo mount --bind /sys /mnt/sys
- sudo chroot /mnt
- (optional, only if you're on Ubuntu/Debian) apt-get install grub-pc
- grub-mkconfig -o /boot/grub/grub.cfg
- grub-install /dev/sdx
- grub-install --recheck /dev/sdx # just to be sure
- Ctrl+D (to exit out of chroot)
- sudo umount /mnt/dev
- sudo umount /mnt
View the status of a multi disk array.
$ sudo mdadm --detail /dev/md1 or $ cat /proc/mdstat
After synchronization of the two disks, the disk label will be the same on both disks.
before sync /dev/sdc # label MX250 /dev/sdd # label WD250 after sync /dev/sdc # label MX250 /dev/sdd # label MX250
If you want to delete an already setup ARRAY in order make a new setup, you have to delete the superblock. In order to delete the superblock, you have to stop the RAID status first. The superblock has its place at the end (about 100 KB) of the disk, so leave the space (about 1 MB), if you partition the disk.
$ sudo mdadm --stop /dev/sdc1 # sdc1 for example only or $ sudo mdadm /dev/md0 -f /dev/sdc1 -r /dev/sdc1 then $ sudo mdadm --zero-superblock /dev/sdc1
If the partitions used in your RAID array are not the same size, mdadm will use the smallest.
To add RAID device md0 to /etc/mdadm/mdadm.conf so that it is recognized the next time you boot.
$ sudo mdadm -Es | grep md1 >>/etc/mdadm/mdadm.conf
Because mdadm.conf will be used in initramfs you have also to update initrd:
$ sudo update-initramfs -u -k all
File system type check
In order to figure out the file system type do:
rudi@rudi-usb:~$ sudo file -s /dev/sdb1 /dev/sdb1: Linux rev 1.0 ext2 filesystem data, UUID=135c7575-a651-4c6f-8a4d-630f97b6691b
Mount /boot in root (/)
If a hard disk is no longer bootable, there is a chance for repair if the file system could be accessed. That can be done with the help of an USB stick with Ubnutu 10.04.1 with -pae (RAID1) support installed. The only difficulty is to merge the separated /boot partition into the root tree. With the following commands it can be done:
Prerequisites: 1. USB-stick 4 GB with Ubuntu 10.04.1 LTS with boot -pae kernel 2. Hard disk which can no longer boot $ sudo mkdir /mnt/sdb3 # create place holder $ sudo mount -t ext3 /dev/sdb3 /mnt/sdb3 $ sudo mount --bind -t ext2 /dev/sdb1 /mnt/sdb3/boot # cover original /boot
Tipps for RAID 1 and Grub2
- If you get an error message because of size differences:
The filesystem size (according to the superblock) is xxx The physical size of the device is xxx Either the superblock or the partition table is likely to be corrupt! do the following: 1. Unmount all partitions $ sudo umount -a 2. Repair the partitions $ sudo e2fsck -f /dev/md2 3. Resize the partitions S sudo resize2fs /dev/md2 $ sudo e2fsck -f /dev/md2
Problem: mdadm: metadata format 00.90 unknown, ignored.
Ref.: http://ubuntuforums.org/showpost.php?p=6345457&postcount=7 Edit: /etc/mdadm/mdadm.conf from: ARRAY /dev/md0 level=raid5 num-devices=3 metadata=00.90 UUID=... to : ARRAY /dev/md0 level=raid5 num-devices=3 metadata=0.90 UUID=...
- Live fix for Grub2 (2010-11-23, 3 Partitions)
Boot from the USB stick $ sudo mkdir /mnt/md1 $ sudo mount /dev/md1 /mnt/md1 $ sudo chroot /mnt/md1 $ grub-install /dev/sdb $ grub-install /dev/sdc $ update-grub $ update-initramfs -u -k all exit, and try rebooting into your install
Shorten RAID1 resync time - DANGEROUS
With my 250 GB 2.5" USB disk it needs about 3 hours to resynch the drives. In order to shorten that to minutes, you can use a bitmap. See also:
http://etbe.coker.com.au/2008/01/28/write-intent-bitmaps/
https://raid.wiki.kernel.org/index.php/Bitmap
Unfortunately the bitmap use gave severe problems while changing disks. One time, my /etc folder got lost. So I gave up on this feature.
$ sudo mdadm --grow --bitmap=internal /dev/md2 $ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid1 sdb2[1] sda2[0] 1550208 blocks [2/2] [UU] md0 : active raid1 sdb1[1] sda1[0] 513984 blocks [2/2] [UU] md2 : active raid1 sdb3[1] sda3[0] 242035200 blocks [2/2] [UU] bitmap: 4/231 pages [16KB], 512KB chunk unused devices: <none>
Setup Postfix Email send
If a hard disk of an RAID array fails it would be helpful to get knowledge about that via email. When installing mdadm the email programm postfix gets also installed. The setup of postfix will usually be done later, NOW.
http://pagewizz.com/Eine-Einfuehrung-zu-E-Mail-Servern/
http://ubuntublog.ch/services/postfix-installieren-und-konfigurieren
http://wiki.ubuntuusers.de/postfix#Authentifizierung-am-Smarthost
http://www.tuxhausen.de/software_postfix.html
http://www.tuxhausen.de/postfix_smtp_auth-3.html#41
http://www.kaischmale.de/de/article/mailversand-mittels-postfix-%C3%BCber-den-smtp-server-der-goethe-universit%C3%A4t-frankfurt/postfix-in
http://www.postfix.org/postconf.5.html
You get a warning if you have not setup an root alias, set it to an existing user.
Especially if you login as a standard user, you should get all root relevant Emails.
WARNING: /etc/aliases exists, but does not have a root alias. Edit file (sudo mc) /etc/aliases and append "root: rudi" (root: <user name>) $ sudo newaliases # to update the aliases data base # If that gives a "permission denied" error, do: $ sudo chown root:rudi /etc/aliases $ sudo chmod g+w /etc/aliases
Important: In order to have the System-E-mail-Name setup right:
Edit /etc/hostname (sudo mc) rudiswiki.de (your domain) restart
In order to restart the semi-automatic setup of postfix type:
$ sudo dpkg-reconfigure postfix You will get a menu selection: Internet-Site Internet mit Smarthost Satellitensystem Nur lokal For the RAID service the selection "Internet mit smarthost" looks most suited, if you have an email account at your provider. Question: System-E-mail-Name : rudiswiki.de <OK> Question: SMTP-Relay-Server : mail.arcor.de <OK> Question: Empfänger von E-Mails an Root und Postmaster: rudi <OK> Question: Weitere Rechner, für die E-Mail akzeptiert werden soll : rudiswiki.de, localhost.de, localhost <OK> Question: Synchrone Aktualisierungen der E-Mail-Warteschlange erzwingen? <Nein> Question: Lokale Netze: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 <OK> Question: Maximale Postfach-Größe (Bytes): 5000000 (5 MB) <OK> Question: Zeichen für lokale Adress-Erweiterung: + <OK> Question: Zu verwendende Internet-Protokolle: alle <OK> Messages, if all is OK: * Stopping Postfix Mail Transport Agent postfix [ OK ] setting synchronous mail queue updates: false mailname is not a fully qualified domain name. Not changing /etc/mailname. # means, no E-mail reception possible - OK setting destinations: rudiswiki.de, localhost.localdomain, localhost setting relayhost: mail.arcor.de setting mynetworks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 setting mailbox_size_limit: 5000000 setting recipient_delimiter: + setting inet_interfaces: all setting inet_protocols: all Postfix is now set up with the changes above. If you need to make changes, edit /etc/postfix/main.cf (and others) as needed. To view Postfix configuration values, see postconf(1). After modifying main.cf, be sure to run '/etc/init.d/postfix reload'. Running newaliases * Stopping Postfix Mail Transport Agent postfix [ OK ] * Starting Postfix Mail Transport Agent postfix [ OK ]
Next the setup for the smarthost:
$ sudo nano /etc/postfix/main.cf # in iTerm with mouse support Check: myhostname = rudiswiki.de (my domain) Also insert below "mailbox_size_limit = 5000000": message_size_limit = 1000000 Append: smtp_sasl_auth_enable = yes smtp_sasl_security_options = noanonymous smtp_sasl_password_maps = hash:/etc/postfix/sasl_password sender_canonical_maps = hash:/etc/postfix/sender_canonical Save the file /etc/postfix/main.cf Create the file sasl_password with: $ sudo touch /etc/postfix/sasl_password and insert the two following lines: # authentication setup for smarthost mail.arcor.de reutexxx:<password> It happens sometimes that the file "sasl_password.db" is not created correct. Therefore it is more safe to restart postfix before creating the data base. $ sudo /etc/init.d/postfix restart $ sudo postmap /etc/postfix/sasl_password A smarthost does need in many cases a valid user email address. For that first create the file "sender_canonical" $ sudo touch /etc/postfix/sender_canonical Then edit that file by inserting 4 lines: # for authentication setup for postfix smarthost access # user name, email address (www-data = web server) rudi reutexxx@arcor.de root reutexxx@arcor.de www-data reutexxx@arcor.de Convert this file to a data base: $ sudo postmap /etc/postfix/sender_canonical Finally restart postfix: $ sudo /etc/init.d/postfix restart Test of the postfix installation (you need the program "mailx" for that): $ mail -s "Betreff: Testmail, from rudiswiki.de" reutxxx@web.de Cc: ENTER type the body- something . # dot for end of the body text CTRL-D # stop input and send email EOT $ tail /var/log/mail.log ... Jan 16 14:33:32 rudi72 postfix/qmgr[7903]: 59022BB21C3: from=<reutexxx@arcor.de>, size=312, nrcpt=1 (queue active) Jan 16 14:33:32 rudi72 postfix/smtp[7955]: warning: SASL authentication failure: No worthy mechs found Jan 16 14:33:32 rudi72 postfix/smtp[7955]: 59022BB21C3: to=<reutxxx@web.de>, relay=mail.arcor.de[151.189.21.116]:25, delay=836, delays=836/0/0.07/0, dsn=4.7.0, status=deferred (SASL authentication failed; cannot authenticate to server mail.arcor.de[151.189.21.116]: no mechanism available) Problem: The authentication failed, in order to debug: $ sudo apt-get install sasl2-bin $ saslfinger -c # for debug ... -- mechanisms on mail.arcor.de -- 250-AUTH PLAIN LOGIN Solution: In file /etc/postfix/main.cf: remove "noplaintext" because the smarthost does not like our encrypted password.
In order to test/debug the mail system do:
Send a test email: rudi@rudi72:~$ mail -s "Betreff: Testmail, from rudi72" reutxxx@web.de Cc: # just ENTER body text 14:59 . # just a dot and ENTER CTRL-D # stop input and send email $ tail /var/log/mail.log ... Jan 17 15:07:46 rudi72 postfix/pickup[13772]: C285EBB2176: uid=1000 from=<rudi> Jan 17 15:07:46 rudi72 postfix/cleanup[13776]: C285EBB2176: message-id=<20110117140746.C285EBB2176@rudi72> Jan 17 15:07:46 rudi72 postfix/qmgr[13771]: C285EBB2176: from=<reutexxx@arcor.de>, size=322, nrcpt=1 (queue active) Jan 17 15:07:47 rudi72 postfix/smtp[13778]: C285EBB2176: to=<reutxxx@web.de>, relay=mail.arcor.de[151.189.21.116]:25, delay=0.47, delays=0.13/0.02/0.08/0.24, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 3A7E810C1A5) Jan 17 15:07:47 rudi72 postfix/qmgr[13771]: C285EBB2176: removed If that does not work: Edit file /etc/postfix/master.cf Append "-v" in line: smtp unix - - - - - smtp -v $ sudo postfix reload postfix/postfix-script: refreshing the Postfix mail system The mail log is now much more extended. ---------------- If the Email function works, it is time to test the mdadm Email: The mdadm Email will be send to user rudi, but not to extern, so change in /etc/mdadm/mdadm.conf MAILADDR reutxxx@web.de $ sudo /etc/init.d/mdadm restart # to apply the change Simple test for email function: $ sudo mdadm --monitor --test /dev/md2 # root / or all arrays $ sudo mdadm --monitor --test --scan ---------------- Simulate a hard disk failure of the swap partition: $ sudo mdadm /dev/md1 --fail /dev/sdb2 mdadm: set /dev/sdb2 faulty in /dev/md1 $ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid1 sdb2[2](F) sda2[0] 1550208 blocks [2/1] [U_] ... Reattach the swap partition $ sudo mdadm /dev/md1 --add /dev/sdb2 mdadm: Cannot open /dev/sdb2: Device or resource busy First you have to stop the array: $ sudo mdadm --stop /dev/md1 mdadm: stopped /dev/md1 Then assemble it again, and the array will be resynced: $ sudo mdadm /dev/md1 --assemble /dev/sda2 /dev/sdb2 mdadm: /dev/md1 has been started with 1 drive (out of 2). ---------------- In case in /var/log/mail/www-data every 5 minutes there is a warning logged from TYPO3, do: $ cd /etc/cron.d $ sudo mkdir typo3 $ sudo mv typo3-dummy typo3/
Spare Hard Disk Preparation
In order to quickly replace a faulty hard disk from a RAID1 array, it is the easiest to prepare beforehand a spare disk.
In my case, with an ASUS netbook with 2 external USB hard disks as an RAID1 array, it is the easiest to plug in a third external USB hard disk. Fortunately the netbook does just have 3 USP ports. The disk is mounted via udev as device /dev/sdd, and must have a capacity >= the RAID1 disk (250 GB).
First partition the new disk with the scheme of the active one:
/dev/sda = internal 8 GB SSD /dev/sdb = first USB hard disk 250 GB /dev/sdc = second USB hard disk 250 GB /dev/sdd = spare disk 250 GB $ sudo sfdisk -d /dev/sdb | sfdisk --force /dev/sdd Problem: /dev/sdd: Permission denied sfdisk: Konnte /dev/sdd nicht zum Lesen/Schreiben öffnen Troubelshooting: $ sudo dmesg | grep sdd ... [55546.398040] Adding 522104k swap on /dev/sdd2. Priority:-1 extents:1 across:522104k $ ls -ls /dev/sdd 0 brw-rw---- 1 root disk 8, 48 2011-01-20 14:37 /dev/sdd Solution: $ sudo swapoff /dev/sdd2 # that does NOT help User "rudi" was NOT a group member of group "disk" Edit /etc/group: disk:x:6:<user> # disk:x:6:rudi logout/login Check with: $ groups rudi adm disk dialout cdrom www-data plugdev lpadmin admin sambashare Neue Aufteilung: Einheit = Sektoren von 512 Bytes, Zählung beginnt bei 0 Gerät boot. Anfang Ende #Sektoren Id System /dev/sdd1 * 63 1028159 1028097 fd Linux raid autodetect /dev/sdd2 1044225 4144769 3100545 fd Linux raid autodetect /dev/sdd3 4160835 488263544 484102710 fd Linux raid autodetect /dev/sdd4 0 - 0 0 Leer Die neue Partitionstabelle wurde erfolgreich geschrieben Power down computer Pull off USB disk 3 Power up computer Plug in USB disk 3 Check with: $ ls /dev/sdd* /dev/sdd /dev/sdd1 /dev/sdd2 /dev/sdd3 Because now there are file systems on the disk, they are auto mounted to /media. Before they could be added to the RAID1 array, they must be unmounted before: $ ls /media 1a94bd26-d57c-4e44-a32b-c183ec832eee/ 91ec8423-41fb-4704-8115-c8588ca3e1a1/ Use TAB for file name completion. $ sudo umount /media/1a94bd26-d57c-4e44-a32b-c183ec832eee/ $ sudo umount /media/91ec8423-41fb-4704-8115-c8588ca3e1a1/
Now it is time to add the spare disk to the RAID1 array, but there is no sync of the data. That will be done in case the spare disk in added to the array.
First try the small boot partition: $ sudo mdadm --manage /dev/md0 --add /dev/sdd1 mdadm: added /dev/sdd1 $ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid1 sdc2[1] sdb2[0] 1550208 blocks [2/2] [UU] md2 : active raid1 sdc3[1] sdb3[0] 242035200 blocks [2/2] [UU] bitmap: 2/231 pages [8KB], 512KB chunk md0 : active raid1 sdd1[2](S) sdc1[1] sdb1[0] 513984 blocks [2/2] [UU] unused devices: <none> $ sudo mdadm --detail /dev/md0 ... Number Major Minor RaidDevice State 0 8 17 0 active sync /dev/sdb1 1 8 33 1 active sync /dev/sdc1 2 8 49 - spare /dev/sdd1
Spare Hard Disk Insertion
Now that you have a spare disk it is time to insert it into the RAID1 array, in order to see if it works.
- Power down the server
- Exchange the second disk by the spare hard disk, look at the status:
$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid1 sdb2[0] 1550208 blocks [2/1] [U_] md2 : active raid1 sdb3[0] 242035200 blocks [2/1] [U_] bitmap: 27/231 pages [108KB], 512KB chunk md0 : active raid1 sdc[1] sdb1[0] 513984 blocks [2/2] [UU] unused devices: <none>
- You see, that /dev/md0 has already synched, but not the other two partitions, so repair:
$ sudo umount /dev/md0 # OK $ sudo mdadm --stop /dev/md0 mdadm: stopped /dev/md0 $ sudo sfdisk -d /dev/sdb | sfdisk --force /dev/sdc $ sudo reboot $ sudo mdadm /dev/md1 --add /dev/sdc2 $ sudo mdadm /dev/md2 --add /dev/sdc3
checkarray every first sunday per month
The installation setup of mdadm includes a setup in /etc/cron.d/mdadm, which does an "--all" check of the arrays at 00:57 every first Sunday every month.
57 0 * * 0 root [ -x /usr/share/mdadm/checkarray ] && [ $(date +\%d) -le 7 ] && /usr/share/mdadm/checkarray --cron --all --quiet From: $ man mdadm (monitor mode): Only Fail, FailSpare, DegradedArray, SparesMissing and TestMessage cause Email to be sent.
Error device or resource busy
It can happen, that you get that error.
mdadm: Cannot open /dev/sdb1: Device or resource busy mdadm: create aborted rudi@rudi-usb:~$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md_d0 : inactive sdb1[1](S) 513984 blocks md_d1 : inactive sdb2[1](S) 1550208 blocks unused devices: <none> rudi@rudi-usb:~$ sudo mdadm --stop /dev/md_d0 mdadm: metadata format 00.90 unknown, ignored. mdadm: metadata format 00.90 unknown, ignored. mdadm: stopped /dev/md_d0 rudi@rudi-usb:~$ sudo mount -t ext2 /dev/md0 /mnt/md0 mount: wrong fs type, bad option, bad superblock on /dev/md0, rudi@rudi-usb:~$ sudo mount -t ext2 /dev/sdb1 /mnt/md0 Other case: Reattach the swap partition $ sudo mdadm /dev/md1 --add /dev/sdb2 mdadm: Cannot open /dev/sdb2: Device or resource busy First you have to stop the array: $ sudo mdadm --stop /dev/md1 mdadm: stopped /dev/md1 Then assemble it again, and the array will be resynced: $ sudo mdadm /dev/md1 --assemble /dev/sda2 /dev/sdb2 mdadm: /dev/md1 has been started with 1 drive (out of 2). dmraid problem: If on an old installation a "dmraid" driver was installed, it can block the "mdadm" devices. Solution is to deinstall the "dmraid" software.
Disk failure simulation
Attention: This method is dangerous for data loss, do at a test system only.
If I pull out hard disk /dev/sdb you will see the following messages:
$ dmesg | tail -n 15 [ 180.892000] usb 1-5: USB disconnect, address 3 [ 182.519534] __ratelimit: 12 callbacks suppressed [ 182.519547] raid1: sda3: rescheduling sector 183725520 [ 182.559386] raid1: Disk failure on sda3, disabling device. [ 182.559390] raid1: Operation continuing on 1 devices. [ 182.559416] raid1: sdb3: redirecting sector 183725520 to another mirror [ 182.564987] RAID1 conf printout: [ 182.564998] --- wd:1 rd:2 [ 182.565008] disk 0, wo:1, o:0, dev:sda3 [ 182.565015] disk 1, wo:0, o:1, dev:sdb3 [ 182.572073] RAID1 conf printout: [ 182.572086] --- wd:1 rd:2 [ 182.572096] disk 1, wo:0, o:1, dev:sdb3 After a reboot with both hard disks again: rudi@rudi72:~$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md1 : active raid1 sdb2[1] 1550208 blocks [2/1] [_U] md128 : active raid1 sdb3[1] 242035200 blocks [2/1] [_U] bitmap: 21/231 pages [84KB], 512KB chunk md0 : active raid1 sdb1[1] sda1[0] 513984 blocks [2/2] [UU] unused devices: <none> repair the array: $ sudo mdadm --add /dev/md0 /dev/sda1 $ sudo mdadm --add /dev/md1 /dev/sda2 $ sudo mdadm --add /dev/md128 /dev/sda3
Mount spare disk for rsync
If you have taken out a USB hard disk from the RAID1 array, it can be used as backup device. But when you boot with an other Ubuntu installation, the RAID1 devices /dev/mdx do not automount, you have to help a little.
rudi@rudi-usb:~$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md128 : inactive sdc3[1](S) 242035200 blocks md1 : inactive sdc2[1](S) 1550208 blocks md0 : inactive sdc1[1](S) 513984 blocks # with one drive only, you have to say "--run" rudi@rudi-usb:~$ sudo mdadm --assemble --run /dev/md0 /dev/sdc1 mdadm: /dev/md0 has been started with 1 drive (out of 2). rudi@rudi-usb:~$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md128 : inactive sdc3[1](S) 242035200 blocks md1 : inactive sdc2[1](S) 1550208 blocks md0 : active raid1 sdc1[1] 513984 blocks [2/1] [_U] mount -t ext2 /dev/md0 /boot # after the rsync procedure exit "chroot" $ exit
RAID 1 FAQ's
Q: What is the version of Grub?
A: $ grub-install -v - should be (GNU GRUB 1.98 1ubuntu5)Q: when installing grub(2) to mdadm devices are you supposed to install it to /dev/sdX or /dev/mdX?
A-2010-11-19: You have to install it on /dev/sdX devices because grub runs before you are able to know that that your are using raid devices. And you should install grub on all active disk that are part of the mdadm.Q: In /etc/fstab after the format parameter ext3 there was errors=remount-ro, what does that mean?
A: In case the file system handler find an error, it will remount the files system in "Read Only" mode, in order to prevent more failures. This gives the opportunity to either fix (fsck.ext2) or to backup the file system. You can not run fsck on a Read-Write mounted file system.Q: Ubuntu version table? http://wiki.ubuntuusers.de/Ubuntu/Releases
Unterstützte Ubuntu Version
Codename
Deutsche Übersetzung
Versionsnummer
Releasedatum
Unterstützt bis
Natty Narwhal
Schicker Narwal
11.04
xx. April 2011
April 2013
Maverick Meerkat
Eigensinniges Erdmännchen
10.10
10. Oktober 2010
April 2012
Lucid Lynx
Heller Luchs
10.04 LTS
29. April 2010
29. April 2013 (Desktop)
29. April 2015 (Server)Karmic Koala
Karmischer Koala
9.10
29. Oktober 2009
April 2011
Hardy Heron
Kühner Reiher
8.04 LTS
27. April 2008
27. April 2011 (Desktop)
27. April 2013 (Server)Q: American special keys on a German keyboard
Sonderzeichen auf deutscher Tastatur mit amerikanischer Tastaturbelegung
Gesuchtes Zeichen
Taste
/ (Slash)
- (Bindestrich, Minus)
- (Bindestrich, Minus)
ß (Esszet)
_ (Unterstrich)
? (Umschalt + Esszet)
= (Gleichheitszeichen)
´ (Tick, Taste neben Esszet)
: (Doppelpunkt)
Ö (Umschalt + ö)
How to speed up the resync process?
- Grow internal bitmaps on the arrays: mdadm --grow /dev/mdX -b internal
UUID's of all devices
In order to keep the overview of all UUID's, you have to document them:
Command example: $ sudo blkid /dev/sda1: UUID="d865e07f-1239-4ce5-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sda2: UUID="a95c611a-03e2-d4d1-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sda3: UUID="c1170d14-3914-bd88-0bfb-90c00f4e6a6d" TYPE="linux_raid_member" /dev/sdb1: UUID="d865e07f-1239-4ce5-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sdb2: UUID="a95c611a-03e2-d4d1-a78e-c32f6eb6df8d" TYPE="linux_raid_member" /dev/sdb3: UUID="c1170d14-3914-bd88-0bfb-90c00f4e6a6d" TYPE="linux_raid_member" /dev/md0: UUID="135c7575-a651-4c6f-8a4d-630f97b6691b" TYPE="ext2" /dev/md2: LABEL="MX250" UUID="624cbccb-5008-40aa-94a1-35d23000b776" TYPE="ext3" /dev/md1: UUID="ad16b658-952b-4276-9623-37d0246c1e06" TYPE="swap" $ sudo mdadm -Es ARRAY /dev/md0 level=raid1 num-devices=2 UUID=d865e07f:12394ce5:a78ec32f:6eb6df8d ARRAY /dev/md1 level=raid1 num-devices=2 UUID=a95c611a:03e2d4d1:a78ec32f:6eb6df8d ARRAY /dev/md2 level=raid1 num-devices=2 UUID=c1170d14:3914bd88:0bfb90c0:0f4e6a6d
Links
- Help for installation from the USB stick
- Help for building a RAID array
http://wiki.ubuntuusers.de/Software-RAID (German)
https://help.ubuntu.com/community/Installation/SoftwareRAID
http://linuxwiki.de/mdadm
https://wiki.ubuntu.com/ReliableRaid
http://www.ralf-schmidt.de/Computer/Howto_RAID1-LVM_Grub.html
http://www.howtoforge.com/how-to-set-up-software-raid1-on-a-running-system-incl-grub2-configuration-ubuntu-10.04
http://www.linuxforen.de/forums/showthread.php?t=269865
http://pith.org/notes/2005/07/23/how-i-fixed-my-raid-1-partition-size-error/
https://raid.wiki.kernel.org/index.php/RAID_superblock_formats
http://serverfault.com/questions/209379/what-tells-initramfs-or-the-ubuntu-server-boot-process-how-to-assemble-raid-array - Alternate hotplug RAID Mirror Install
Help for Grub2 Installation from Live CD
http://grub.enbug.org/Grub2LiveCdInstallGuideUbuntu Grub2 documentation: https://help.ubuntu.com/community/Grub2
Help for booting Ubuntu 09.10 - 10.04 (German)
http://wiki.ubuntuusers.de/booten- Grub2 documentation
http://wiki.ubuntuusers.de/GRUB_2/Terminal_Befehle (German)
http://wiki.ubuntuusers.de/GRUB_2/Reparatur (German) - Good explanation for mdadm setup and repair
- Official Ubuntu documentation for RAID setup
https://help.ubuntu.com/10.04/serverguide/C/advanced-installation.html
- chroot - Live CD-ROM
- Problem with UUID's, device - filesystem
http://ubuntuforums.org/showthread.php?t=1503641 http://forum.ubuntuusers.de/topic/raid1-md0-wird-zu-md-d0-und-laeuft-nicht
List of pages in this category:
-- RudolfReuter 2010-12-19 13:02:01
Go back to CategoryHowTo or FrontPage ; KontaktEmail (ContactEmail)