Adding two SATA SSD devices to a RAID10 six SATA array.

I needed a bit more space on my NAS’s RAID10 array which was 6 x 2TB drives.To be honest iI’m not sure why I am using RAID10. The array was initially on my main Linux workstation. Then I decided I needed a separate NAS as my existing NAS was way too small, so I moved it across. RAID5/6 would give me more space. But I guess I like the flexibility and the ability to survive failure of two disks even though it does reduce space by 50%!

The server is running Debian and is headless. I know there’s loads of NAS OSs. But I do prefer to do things myself. The boot/root partitions are on a single SATA SSD card and the (now) eight drives are plugged into a Seagate Smart Host bus Adaptor H240.

I found two “consumer” Crucial 2TB SSD disks on Black Friday for £65 which seemed reasonable. I did wonder how well two SSDs would do in a RAIDarray with spinning drives. Let’s find out….! So this is what I did (which I am blogging about so I do not have to remember next time!). Interestingly the last time I blogged about growing a RAID array was quite some time ago. Also that’s the only time I’ve ever got comments on my blog (127 to be exact!). I think growing RAID arrays with MDADM was quite new back then.

The procedure to add new devices and grow the raid array is:

Procedure to grow an array with two new disks

  • physically add new disks
  • Partition
  • add disks to array
  • increase number of active disks in array to grow the array
  • grow filesystem

The new disks are /dev/sda /dev/sdd

Partition

For GPT use sgdisk to copy the partition table from one disk to a new one

Backup first of course!

sgdisk /dev/sdX -R /dev/sdY

sgdisk -G /dev/sdY

“The first command copies the partition table of sdb to sda/d

sgdisk /dev/sdb -R /dev/sda

sgdisk /dev/sdb -R /dev/sdd

Now randomise the GUID of each device:

sgdisk -G /dev/sdd

sgdisk -G /dev/sda

Add new devices

mdadm --add /dev/md1 /dev/sdd1 /dev/sda1

mdadm: added /dev/sdd1

mdadm: added /dev/sda1

These are added as spares as the number of active devices does not change. Let’s check:

# cat /proc/mdstat
Personalities : [raid10] [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4]
md1 : active raid10 sda1[8](S) sdd1[7](S) sdc1[5] sdg1[0] sde1[6] sdh1[3] sdb1[1] sdf1[4]
5860141056 blocks super 1.2 512K chunks 2 near-copies [6/6] [UUUUUU]
bitmap: 4/22 pages [16KB], 131072KB chunk

Increase number of devices to include new ones

mdadm --grow --raid-devices=8 --backup-file=/mnt/USB/grown_md1.bak /dev/md1

The –backup-file creates a backup file in case the power goes. Not essential as I have a UPS. Also the filesystem is still mounted. However, to speed it up I turned off all services except the DNS/DHCP server. The less disk activity the quicker the reshape will finish.

The reshaping took about 20 hours. Much less than I thought.

Now we need to resize the filesystem. Unmounting is not essential for growing a ext4 filesystem (although it is for shrinking), but hey it’s a lot safer so I shut off everything and unmounted it

systemctl stop smbd
systemctl stop docker
umount /mnt/storage

resize2fs /dev/md1

This gave an error that the filesystem needed checking first.

e2fsck -f /dev/md1

resize2fs /dev/md1

This took about 30 minutes.

Finishing off.

Now let’s get it all back up and running.

mount /mnt/storage

systemctl start docker

systemctl start samba

systemctl start smbd

systemctl start docker

systemctl restart docker

The entry for the mdadm device does not need updating. Previous it did but I think that’s when I was using the 0.9 metadata block.

mdadm --detail --scan

cat /etc/mdadm/mdadm.conf

One bizarre issue was that when I restarted all the docker containers they downloaded a new image rather than using existing images. I have no idea why that happened.

Plan9 – what I do after an install

This was written in Markdown using acme on Plan9 with an sshfs mounted folder of my NextCloud setup (via a Linux host). Tidied up in Drafts, then posted on macOS using Marsedit! This post is for me really. ..it will also be a WIP.

initial post install stuff

Create a new user

Everybody seems to use the default user, Glenda (from Plan9 from outer space). This is the equivalent of a root* unix user. Except I do not like that.

*Well almost. There is no root user that is all powerful in plan9

This also shows how you run a lot of command stuff by cat’ing text to the running process. cwfs.cmd is the file server process that runs the CWFS filesystem (it’s different if you run another filesystem). You can do this in one step with the “con” tool. But i prefer to do each step one by one as I remember it better.

echo newuser chris >>/srv/cwfs.cmd

You also need to add the new user to the sys and upas groups.

echo newuser sys +chris >>/srv/cwfs.cmd

echo newuser upas +chris >>/srv/cwfs.cmd

Various how to pages also suggest the “adm” group too. I think this is equivalent to root. I have not yet added my user to it and not found any errors.

term% cat /adm/users
-1:adm:adm:glenda
0:none::
1:tor:tor:
2:glenda:glenda:
3:chris:chris:
10000:sys::glenda,chris
10001:map:map:
10002:doc::
10003:upas:upas:glenda,chris
10004:font::
10005:bootes:bootes:

I’ve no idea what the other groups are for!

Then reboot, login as this user. You will see errors. But run the newuser script to setup your $home folder and profile


/sys/lib/newuser

customise plan9.ini

This requires you to mount the 9fat filesystem which contains a plain text file (hey everything is plain text file!) that configures the boot process.


9fs 9fat
cd /n/9fat


“9fs” is equivalent to mount and “9fat” is the partition.


Nice and simple.


You now need to familiarise yourself with a text editor. Acme is the best. I have found myself having to use “ed” a streaming text editor. It’s Ok, but very basic and quite painful to do anything other than simple emergency edits. Keep backups!

acme plan9.ini

Setup plan9.ini to boot straightway into my user account

Once you have your user then you can bypass which filesystem to boot and which user to boot into.
Remembering that at anytime you are promoted for options in the boot process then you can type “!rc” at a prompt to launch a minimal terminal to fix the issue. ..and booting from the USB install image always allows you to easily mount the 9fat partition and fix things.

change the video driver to IGFX

customise your desktop – rio

Rio is the windowing application.

To customise Rio you need to edit your profile

acme $home/lib/profile

I now configure my profile to load a “$home/bin/rc/riostart”, which autostart a few tools and a few “rc” shells.
Change

rio

to

rio -i riostart

Mine is:

#!/bin/rcwindow 0,0,161,117 stats -lmiscewindow -miny 130
window bar
# run a system shell on the serial console
~ $#console 0 || window -scroll console`

Bar is a cool little tool you will have to compile and install.

Change the font

Change the following line. There are many different fonts in /lib/font:

#font=/lib/font/bit/vga/unicode.font
font=/lib/font/bit/dejavusans/unicode.14.font

keyboard

Add this line just before loading rio:


cat /sys/lib/kbmap/uk > /dev/kbmap

misc

Setup SSH

The instructions are here. However, there is an omission in that first line as “role=client” needs to be added.

auth/rsagen -t 'service=ssh role=client' >$home/lib/sshkey # generate private key
auth/rsa2ssh $home/lib/sshkey >$home/lib/sshkey.pub # generate public key, if you need to share it
cat $home/lib/sshkey >/mnt/factotum/ctl # put the private key in the password manager
echo 'ssh sha256=DDDDDDDDDDDDDDDDDDDDDDDD server=scotgate.pixies' >> /usr/chris/lib/sshthumbs

a few useful tips

Mounting a linux filesystem over SSH

sshfs is useful.

sshfs chris@scotgate.pixies
cd /n/ssh

..will mount your $home folder on the remote host as /n/ssh

Note if you are using Drawterm then /srv/ has your root filesystem of the host already.

Software to install

Treason

River Cam gauge

In a break from my usual plan9ing here’s the gauge for the lock just upstream of us on the River Cam from Shoothill River Gauge map. I had the boat turned around facing downstream as I was cleaning it last week. The back of the boat with its rudders and props is much more vunerable to damage from debris. With the biblical rain yesterday I saw the river had risen and dropped overnight (mud was deposited). Generally when that happens then 24 hours later the river rises again as rain from upstream reaches us. The river did not look to be running that fast, so I decided to turn the boat around by ropes in anticipation of the stronger flows to come. Whoops. The river flow was quite deceptive. It was a real struggle to spin the boat around. All done safely now.

So here’s a gauge for the river flows.

This is the code.

<iframe frameborder="0" width="300" height="446" src="https://gaugemap.blob.core.windows.net/gaugemapwidgets/1769-1909-3-300x446.html"></iframe>

The EA/.gov site says the level is normal albeit still rising:

River Cam level at Cambridge Baits Bite

Plan9 continues

So Plan9 is becoming a very large time sink! After playing with a VM/QEMU I decided to buy a cheap minipc and install on some real hardware (Lenovo ThinkCentre with 16GB Ram and an i5 – £70 from eBay).

Installing the 9front fork of Plan9 is really easy. I did decide to complicate matters and use two drives so I had to manually partition them as the install script is quite simplistic (but really does make life easier!). Plan9 is a distributed OS. It needs a CPU server a file server, and auth(entication) server and then clients. Due to it’s quite ancient heritage the file system (CWFS) uses a cache and a WORM (write once read many) partition. Which in times gone by could have been an optical drive jukebox. As the name suggests this partition is read from, but writes are instead done to the fscache drive.

Other partitions are:

  • 9fat (DOS partition where the boot parameters are stored in a file called plan9.ini)
  • other
  • nvram (used to simulate a real NVRAM storage where authentication stuff is stored).

Drive/partition naming is very similar to linux

/dev/sdE1/fscache
/dev/sdE0/fsworm

So for some inane reason I put the WORM partition on one largish SATA spinning drive and the rest on a different drive. Of course I made life harder for myself. plan9 prides itself on a simple filesystem. It does not really have command completion as the philosophy is that files should not be squirrelled away in hard to remember locations (you do have simple filename completion with CTRL – f). Not sure I quite agree with that, but what it does is automatically find portions and mount then (no stab needed here). But of course since my main partitions are on a separate drive which I went and attached to the second SATA port, then it cannot find them so I do have to specify them in plan9.ini

9fs 9fat
cd /n/9fat
acme /n/9fat/plan9.ini

(More on acme later. A quick to learn but wacky text editor).

Of course I found this out the hard way with an unbootable system in configuring the system to allow remote connections (see next section). But a quick reboot from the install USB stick. I mounted the 9fat partition. Unfortunately without a graphical environment none of the text editors worked. But sed does. A quick reboot later and all was well. But now I do create backups when I edit plan9.ini!

Now how do I connect to this headless plan9 server. With Drawterm of course! Drawterm is an emulated plan9 client. Calling it emulated is not really accurate as it runs natively on whatever OS you install on (for me that was Linux) and it brings the 9p protocol to connect.

But to do this you need to configure the authentication server and allow remote connections

Anyway with the networked OS model I got Drawterm working on Linux and the CPU, file and auth servers are all running on the ThinkCentre.

So after quite a few hours I find myself with this!

Drawterm on the right, Plan9 under QEMU on the left

Plan9 running under QEMU, with Drawterm on the right connected to a real Plan9 server.

Spot the difference?

No, me neither!

Taking procrastination further – Installing Plan9 under QEMU

So instead of doing something useful and needed (like gardening!) during a few hours off last week I decided that what I really needed to do was install plan9!

A few months back I got various Windows NT VMs and OS/2 VMs running. Then Haiku (a port of BeOs) the other week and now this….!

I started with the last “official” build of Plan 9. This was difficult and I had to abandon it as I just could not get it sufficiently configured to launch Rio (the window manager). However, further googling shows that there is a port of plan9 called 9front. This is under continual development and the build I installed was just a few months old. Under QEMU (using libvirtd) it was a pretty hassle free install.

Once installed I found that there is also a recent port of NetSurf, as the two existing browsers are a few decades behind. Downloading the sources from Git, compiling and installing was quite straightforward. The only hurdle was that text does not auto scroll in terminals. Not a significant hurdle you may think. But I guess due to “everything is a file” when the text reaches the end of the terminal then the process stops working. Here as soon as the build script got to the bottom of the terminal it just stopped compiling. Luckily there is an auto scroll option.

  • Middle click and choose SCROLL/NOSCROLL
  • Incidentally you really need a three button mouse. For example to open a new terminal:

  • RIGHT CLICK
  • Choose NEW
  • The cursor changes to a cross. With the RIGHT MOUSE BUTTON draw where you need the new terminal to be.
  • Screenshot from 2023 09 08 14 18 18

    Screenshot from 2023 09 11 12 31 52

    I have no idea what I will do with it. But that goes for more or less all the VMs I have. Install, fire up a few times, then never again.

    Indulgence

    Wagyu entrecôte steak

    So on the way back from holiday and staying with relatives for a night in Brittany. We ordered a properly indulgent Wagyu entrecôte. Was very tasty and the meat just melted in your mouth. But the price did make me think I prefer a tougher and tastier cut.

    First lot of summer cabbages.

    I planted twelve savoy and twelve red cabbages back in March. These were plug plants as I’m too lazy to grow from seed.

    I kept them under mesh and did some weeding (although not a lot).

    Well it’s time to cook them.

    IMG 8861

    trouble is by the time I’ve taken all those half eaten outer leaves the heads are quite small! Hopefully also tasty!

    IMG 8862

    Here’s it cooking. Smothered Cabbage. A New Orleans staple that I’ve never tried before.

    IMG 8863

    Sinead O’Connor RIP

    I’m not generally one to publicly mourn the deaths of celebrities. I generally never knew them and so whereas I will miss their art/science/whatever, I cannot really feel grief (I’m talking for myself here). But there’s a few celebrities that perhaps I can grieve. John Peel, Mark E Smith, Bowie and now Sinead O’Connor.
    She’s the same age as me but it’s not that. It’s that her music, her bravery, her crazy comments and never, ever giving a fuck was just so right. From Mandinka to Queen of Denmark, from ripping up a photo of the Pope (apparently her mother’s photo that was hung in their home) to telling Prince to fuck off.
    She had a “lively” period on Twitter a few years ago. Sometimes poignant, but generally funny and always personal.
    She did a hell of a lot more than that Prince cover. I can appreciate that song, and the video was wonderful, but, and this may sound quite petty but I would think Sinead would never have replace “you” with “U” in a song. Her songs were far better than that.
    My favourite and the first song of Sinead’s I ever heard was Mandinka. It still sounds good even though it’s almost forty years ago since she recorded it (I think!). But her albums always stayed fresh. “How about I be Me (and you be you) (2012)” is a wonderful album and “Queen of Denmark” from that album is so full of emotion and brilliance.
    Her third album “Am I not your Girl?” is also superb. The one she was supposed to be promoting on Saturday Night Live when she instead tore up the photo of the pope. I listened to that album over and over again for quite a few years. Which as a covers album I would generally never do. I listened to it twice through last night. Still as good.

    old operating systems

    So despite gorgeous weather and a load of gardening jobs that I really need to get on top of, I spent quite a bit of Sunday playing about with libvirt, QEMU and a bunch of old operating system ISOs.

    I’d played with some downloaded OS/2 VMs before, but could never got them running under KVM/QEMU. They were all either VMWare VDI/VMDK images or Virtualbox ones. QEMU can convert these into QCOW2 (QEMU’s native format) but whereas I got OS/2 Warp 3.0 running fine. Warp 4.0, 4.52 eComstation VMs all failed to run. However, it turns out that the ISO of OS/2 Warp 4.5.2 is bootable and an installation (I never bothered with 4.0) was surprisingly easy.

    I even got OS/2 2.0 running fine.

    OS2Warp3 0 Warp4 52 runningQEMU

    I also tried with Windows NT 3.5 and 4.0. Both installed nicely.

    WinNT3 51 NT4 0 runningQEMU

    There are many instructions online so I will not bore you with them anymore (although I should add some links).

    I moved from DOS/Windows 3.1 straight to OS/2 Warp 3.0. When I left work and went to college in 1998 at the Freshers Fair there was a goodie bag that included a free licensed CD of OS/2 Warp 3.0. I installed it and never went back. I was even a founder member of WarpUK, a group of enthusiasts. I’d link to them but no trace exists online. Although I do have a few of our CDs. I went to Warp 4.0, then eventually gave up on OS/2 and moved to Linux (Ydraggasil, then Debian and onto Gentoo which I still run now).

    I used Windows NT 3.5 quite a bit. I think I had a dual boot system with OS/2 although to be honest I cannot really remember. I installed but never really used NT 4.0.

    I’m not sure I miss the joys of downloading 20 odd 1.4MB floppy fixpack images, “burning” to actual floppy discs and updating the OS…! But I do miss OS/2 sometimes! A great OS.

    Sanding down an old oak table

    When we were given a lovely dinner table and chairs, our old round oak table was moved to the veranda for almost alfresco dining. Even though it’s protected from direct rain it has deteriorated over the years and so it was time for a refresh.

    So I sanded down and oiled.

    TableSanding

    I used a belt sander, starting with 40 grit, then 80, 120 and 240. A belt sander is not ideal for furniture but it’s all I have.

    For the legs I just used a 1/3 sheet sander and went from 80 to 120 then 240.

    the legs of a table freshly sanded

    The top was finished with five coats of Tung oil, starting with 10% oil to 90% white spirit and finishing with pure Tung oil.

    Tung oil is the main ingredient of many other oil products. It’s from the Tung nut and is food safe (unless mixed with solvents) and is ideal for finishing work surfaces in kitchens. The main disadvantage is that is takes a looong time to dry (the 5th coat of pure oil took four days to become touch dry and probably much longer to fully cure!). In fact it does not dry and instead cures. It needs oxygen to do so and will fully polymerize unlike other furniture oils, so is pretty much waterproof. One of its other advantages is that the end result is matt and if you need to repair a scratch or damage, then you can just wire wool a section and re-treat. The end result is generally a seamless repair. Which is pretty much impossible with other oils or varnishes without a lot of sanding work! I use tung oil mixed with solvents, to aid the initial soaking in of the oil.

    a freshly sanded oak table partly oiled.

    a freshly oiled oak table

    The legs just had two coats.

    The final result.

    An oak table that has been sanded and oiled on a dust sheet over terracotta tiles.

    Here’s to lots of drinks and food over the summer (wait where’s the sun?).