Ferg's gaff – Comics, Macs, RAID, Linux, Gentoo, some science and a dash of atheism. No longer a wet lab molecular biologist, I now work for MacVector, Inc.

March 21, 2023

Before we had a bathroom

We’ve got plumbers in today replacing a few items that have broken in the 15 years since we fitted them. Before we moved into this house we had about 11 months of renovation to turn a flimsy house with no insulation, minimal plumbing and wired with appallingly dangerous wiring (rubber coated cables where the insulation had disintegrated), into something that could be lived in.

I could rant about how the plumbers back then made absolutely no thought as to future maintenance, but I will not. However, in looking for how the shower was fitted (so it can be replaced) I came across this photo before the bathroom even existed.

02062009449

In this photo the flat roof of the utility room below has been removed and replaced by joists to provide a floor. The door opening was previously a window at the top of the stairs. The shower was fitted between the two studs you see to the left and in front of the white door.

Here’s how it looks now from the exterior (the window you see is about where I was stood when I took the above photo 14 years ago.

March 17, 2023March 18, 2023

A years old Wifi issue finally sorted! Yeah!

I’ve always wondered why I had wifi issues in my garden office on my phone and more generally everywhere else in my garden. It’s only 10m away from an access point with good quality external aerials. Which should give decent coverage over the entire garden and offices.

I have a central Draytek 2820n router connected to a Netgear GS724t switch (then another in my garden office). There are two Draytek Access points connected to the switch. AP-800 and AP-900. Both legacy products now, but OK.

Wifi reception in the house is generally decent. But in the garden and our garden offices it is rubbish. There are multiple Smart switches that are always disconnecting (in fact I threw one away a few years ago and replaced it with a new one). A pair of Sonos speakers that are always disconnecting (though they should be using SonosNet and not general wifi) and a Raspberry Pi on the boat that I can rarely reliably SSH to.

Our garden has some overhead power cables above it. These are 11kv with three phases and power the entire village.

Yesterday morning I just noticed that I was getting a very decent wifi signal on my phone (20Mbps down). Then it dawned on me that the cables are turned off today for tree work and the village is being powered by a diesel generator for the day. The naive “jumping at straws because this dammed wifi problem is never going away” person in me immediately assumed that interference from the cables was the issue

Then more knowledgeable people than me told me that a 50hz electricity cable will not interfere with a 2.4/5Ghz wifi signal. The frequencies are too different. Still something is awry. I also noticed that the three smart plugs in my garden that are generally problematic are connected without issue. Same goes for two Sonos speakers. I could even connect to the RPi with a decent connection. Enough to update Raspbian without having to use Tmux.

However, then the connection fails again and the power has yet to be restored to the cables. So what is the problem?

The Draytek router has AP central management and I configure the two Draytek access points using that tool. I’ve rarely looked at the config pages of the actual APs.

Draytek CentraAPmanagement

But now I looked at one. Turns out that it was using two wifi networks with the same SSID.

The AP in the house had both SSIDs connected to LAN-A, but the AP in the garden had one of these “pseudo” networks connected to LAN-B. The LAN-B is not connected to anything!

DraytekAPdualSSIDs

So when the power went off the APs rebooted. Then the devices get a decent connection. But as the phone roamed it switched to the SSID with no network connection and so the iPhone goes crazy with no internet connection and switches to 4G. I have no idea why the Smart switches disconnected though.

I just turned this off on both APs and everything now connects flawlessly with a strong signal. I have had this problem for years…! Sheesh…!

UPDATE March 18, 2023
So the wifi failed again. I went back to check and the multi SSID option had been re-enabled. Turns out the AP Management tool on the router has a profile that enables it, and it kept cloning it to the access points. Now that’s fixed. Fingers crossed.

March 13, 2023

migrated blog to new hosting provider

So after moving my old hosting provider to a monthly renewal 12 months ago in preparation for moving it. I finally found a day to migrate it all over to a new provider on Saturday. I chose Mythic Beasts. A Cambridge based company that I have a few other domains hosted on and have been impressed by their plain talking sales, sensible pricing and proper technical support.

Their support pages suggests a “no downtime” workflow that involves copying their DNS entries for your new site to your existing hosting provider’s DNS record. So even though you do not know when the changes propagate through DNS servers both your existing provider and the new one’s all point to the same new server on your new provider’s site. Very straightforward.

I also discovered a cool WordPress plugin (Updraft Plus), that I’ve been using for backups for some time, has a very restore function that you can easily and quickly restore a backup to a new site. If I’d have known how easy and reliable this perhaps I would not have bothered exporting my site, and importing to a backup server. Just In Case.

March 7, 2023March 7, 2023

Spring is coming – flowers in the garden

Our garden was well planned by the previous owners. They bought the house in 1933 and spent their lives planning the garden. No lawn, but vegetable patches in the middle and flowers around the borders. We have left the garden much wilder. The vegetables patches were replaced by grass (although we reverted part of that to vegetables patches during the first lockdown). The borders are still full of flowers. All planned so that the last weeks of Winter and Spring are full of flowers, snowdrops, aconites, crocuses, daffodils, then tulips, crown imperials, irises and plenty more. Here’s the first batch.

(The first daffodils photo is from elsewhere in the village, but it’s a lovely photo!).

February 28, 2023February 28, 2023

Goodbye to my boots!

So I say goodbye to my Lowa walking boots this morning. I’ve had these many years and they have really stood up to abuse. The left boot developed a leak in the heel that stopped them being used too seriously. But there again since my daughter was born serious hikes have been rare. Still I kept them as they feel like trainers but are very firm. Also the hinged ankle protector makes it impossible to twist your ankles regardless of the angle you put your foot down.

But in clearing out the “solar thermal” cupboard I realised the material in the sole had rotted way. unrepairable so into the bin they go.

February 20, 2023

half term holidays

So the plans for the start of half term were well and truly skewered by covid finally catching up with me! it was quite mild apart from two sleepless nights, but no venturing anywhere until those parallels lines became singular.

We had our first outdoor dining experience of 2023. Naan breads and sausages cooked over a fire.

A visit to the nearest pub (Crown and Punchbowl) then ham, egg n chips at the newly reopened Plough and Fleece (how lucky we are to have two pubs in Horningsea).

To round off the week we stayed overnight in an Airbnb in the middle of a wood. Really lovely new wood cabin in the grounds of Houghton Hall.

The surrounding woods were lovely. Within the space of a twenty minute dusk walk we were circled by a barn owl hunting, watched by a single deer, and saw the hind legs of two hares as they scarpered.

The next day I had the nicest bacon and egg sandwich I’d have for many years then a lovely walk the next day on the beach at Holme Near Sands rounded off our 24 hr trip to Norfolk. Our tradition of always calling at Brancaster Staith for some dressed crab was spoilt somewhat as the van was not there. But it was saved by a roadside place selling fresh mussels. So our dinner on our return was moules marinere. Yummy!

February 15, 2023

First outdoor cooking of the year

It’s school half term, so whereas it’s still pretty damp outside, it’s a good time to cook sausages and naan bread on a campfire outside.

I’ve learnt from previous years and rather than having to turn a naan regularly so it does not flow through the grill bars, I used a baking tray over the fires instead. It still needed turning quickly to avoid burning, but they baked nicely and were the yummiest I’ve done yet!

The grill is made from the steel bars of an old scrap iron bed frame at the right width to slide a grill from an oven. Fit’s nicely across the fire pit.

February 13, 2023

Migrating My Root Drive Raid1 Array to a Pair of NVME Drives.

I’ve always tried to keep upgrading my own Linux boxes. I enjoy it and as I found out a few years ago if I do not regularly keep updating hardware then I completely lose knowledge of doing so.
My latest project is to move all the services from my workstation to a separate box. I’ve got a few RPis around the house running a few services, but I’ve always used my workstation as a games machine/server/development/everything else. In fact for a number of years I stopped using Linux as a workstation at all and this machine was a headless server.
Because of this cycle of continuous upgrades this computer has existed for probably twenty years. Always running some form of Linux (mainly Gentoo).
Currently it’s using some space heater of a server motherboard with a pair of E5 2967v2 on a Supermicro X9dri-LN4 motherboard with 128Gb of ECC DDR3 RAM (very cheap!). That’s fine over winter (my office has no heating) but I do need to fully transfer all the services to the low power Debian box under my desk instead!
Anyway this computer boots from pair of SATA SSD drives in a RAID1 array, with a six disk RAID10 array for data. That array needs to be replaced by a single large drive when I’ve finished moving services to a new machine….!
The motherboard is too old to EFI boot from NVME drives. However, whilst browsing Reddit I came across some people talking about using an adaptor card to add in four NVME drives and using bifurcation toggle each drive proper access to the four PCIe channels that NVME devices need. So x4,x4,x4,x4 instead of x16.
This was not supported on this board, but turns out Supermicro did release a new BIOS that does support bifurcation.
So I bought the card they suggest and a pair of 1TB NVMe drives. The drives are only PCIev3 as that’s all the motherboard supports. PCIe is backwards/forwards compatible. but PCIev4 drives are considerably more expensive than PCIeV3 ones. I may as well get a pair of these, then when I upgrade to a PCIeV4 motherboard the available drives will likely be larger and cheaper!
– Asus M.2. X16 Gen 4
– 2 x WD Blue SN570 NVMe SSD 1TB
The adaptor and cards came. The adapter has got a lovely heatsink that sandwiches the drives in with a small low noise fan.
The adaptor took ten minutes to install. When booted up the BIOS setup disk was a little tricky to enable bifurcation as the slots are numbered from the bottom. This one was CPU1/slot 1.
I had to recompile the kernel to add NVME support, but once booted the pair of drives were there.
After many, many years of using /dev/sdX to refer to storage devices (I was using SCSI hardware before SATA), it does seem a little strange to be running parted on /dev/nvme1n1 then getting partition devices like /dev/nvme1n1p2
I know I should likely move to ZFS, but I’m knowledgeable enough about mdadm not to completely mess up things! ..and replacing a pair of RAID1 devices is just so easy with mdadm.

Workflow is:
– partition drive.
– add to raid array as spare
– fail drive to be removed and then remove it.
– Wait until raid1 array is synced again.
– repeat with second drive
– resize array, then resize the filesystem

procedure

fdisk /dev/nvme0n1

We can use fdisk again as fdisk is GPT aware. Previously we’d always used parted. But I prefer fdisk as I know it!
– partition
– Label the drive as GPT
– Make 256mb partition and mark as EFI boot.
– Make 2nd partition for the rest of the drive and mark that as type Linux raid.

Now add that drive to our raid1 array. For some reason it was not added to a spare, but was instead immediately synced to make a 3 drive raid1 array. I think this is because I previously created this array as a three drive array (for reasons I forget). I guess that’s stored in the metadata of the array.

mdadm /dev/md127 --add /dev/nvme0n1p2

We can watch the progress:

watch cat /proc/mdstat

Once completed we can fail and then remove the drive

mdadm --manage /dev/md127 --fail /dev/sdh3
mdadm /dev/md127 --remove /dev/sdh3

Then let’s update our mdadm.conf file

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Then remove the oldlines:

vi /etc/mdadm/mdadm.conf

Finally let’s wipe that old drive of RAID data so the array does not try to reassemble it to the array.

wipefs -a /dev/sdh3

A reboot is a good idea now to ensure the array is correctly assembled (and new PAT reread).

Now let#s copy the partition table (PAT) to the second new drive.

sgdisk /dev/nvme0n1 -R /dev/nvme1n1

Then randomise the UIDs

sgdisk -G /dev/nvme1n1

Check all is OK

Now repeat adding the second new drive:

mdadm /dev/md127 --add /dev/nvme1n1p2
mdadm --manage /dev/md127 --fail /dev/sdg3
mdadm /dev/md127 --remove /dev/sdg3
mdadm --detail --scan >> /etc/mdadm/mdadm.conf
wipefs -a /dev/sdg3

resize after adding new devices

mdadm –grow –size max /dev/md127
df -h
Then resize the filesystem
resize2fs -p /dev/md127

benchmarks

dd if=/dev/zero of=/home/chris/TESTSDD bs=1G count=2 oflag=dsync

2+0 records in
2+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 4.7162 s, 455 MB/s

dd if=/dev/zero of=/mnt/storage/TESTSDD bs=1G count=2 oflag=dsync

2+0 records in
2+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 10.0978 s, 213 MB/s

dd if=/home/chris/TESTSDD of=/dev/null bs=8k                                                              !10032

262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 0.666261 s, 3.2 GB/s

dd if=/mnt/storage/TESTSDD of=/dev/null bs=8k                                                             !10034

262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 0.573111 s, 3.7 GB/s

I did think that the write speed would be faster. But I guess dd is not the most accurate of benchmark tools.

February 8, 2023February 8, 2023

Our hidden little coal burning range

Inside an old house in the process of renovation. Walls are bare brick and scaffolding is holding the roof up.

One of my lockdown projects was cleaning up an old range we found in the house. This blog post has been in Drafts ever since.

When we bought the house about 16 years ago. We bought it from an old couple who had moved into a home (we never met them). They had bought the cottage in 1933 and lived there ever since. They were 102 and 98 yrs old.

When they bought the cottage they renovated it. Removing the thatch and adding an extra floor.
We knew it was an old cottage, however, we assumed (and the survey suggested this too) that all the old features had been “modernised” and removed.

Here’s how the cottage looked originally (around the 1890s).

Shortly after the “modernisation” in the 1960s (photo taken 1970).

Here’s how it looked when we purchased it 18 years or so ago.

..and here’s how it looks now. Photo just taken after I’d spent three months putting eight coats of limewash on the lime render.

But when we started renovating the cottage we found lots of old original features that had had just been covered up. Two lovely features that we discovered are an inglenook fireplace and our little coal burning range in the kitchen. Both were completely hidden before we purchased and it was only when we started ripping down fibreboard covered walls that we discovered both.

Just after I’d started ripping of the fibreboard around the tiled fireplace, and I discovered the oak bressumer and the inglenook fireplace.

After
The kitchen looked like this when we bought it:

(The water tank is where that little range is now)

When I removed the water tank and associated crap. I found an old range. Seemingly destroyed with all the doors missing.

But when I cleared all the rubble away I found all the missing bits.
A crappy brick wall was removed, cement render removed and repointed in lime and sand.
I put a wine rack on top of it, meaning to go back and clean it up. I’ve only just done so.

Incidentally the 85 year old son of the owners visited here 8 years ago. He grew up in this house and he has never seen that range.

There is still burnt coal in the range, but the chimney above is missing. It looks to have been torn down when they reroofed it many years ago.

February 2, 2023February 2, 2023

A review of 2022

So a quick review of 2022. Not done one for at least a decade.
I started writing this during Christmas, then a few updates. I wanted to add links and photos. But hey, it’s now February and I have not. I should learn from this. I either blog lightly, or not at all!

So I got more and more into regularly gaming. From starting again after not playing anything since Quake (the first), during the pandemic, to this year regularly playing games, and even treating myself to a Steam Deck (which is wonderful). Witcher II, Fallout 4, Elite Dangerous, God of War, and of course Minecraft with my daughter. I’ve got her into gaming as well. Starting the year with games that she’d played on Apple Arcade, Sneaky Sasquatch, then moving onto Minecraft with me (and others), then onto the Steam Deck. She loves Alba, Sonic, Stray and Slime Rancher. Other games she’s tried but failed to get into are Planet Zoo (which her cousin plays),

Apart from a new GPU (see above for the reasons, AMD RX 6600XT) and Steam Deck (also see above), then I’ve not done hardly anything. no RPI’s Arduino’s or anything properly geeky. However, we did upgrade to Gigabit FTTP which is very, very cool. I also switched my office switch from a dumb Netgear 16 port GS116 to a Netgear GS724t. Old tech, but quite reliable (and at £20 on Facebook quite affordable!). This is linked with two aggregated links between the house switch. Same router but an earlier hardware version.

I did switch most “home services, TVHeadend, Minecraft server, Motion cameras and a few other trivial things to Docker. I also moved these from my Gentoo workstation to a new headless Debian box made from bits and pieces I had lying around (well I did upgrade RAM and CPU from eBay!). I guess on that topic I also finally switched from Mythtv to TVHeadend, when I added a second satellite dish as I found that all French TV services are broadcast unencrypted on multistream feeds from the 5W satellite. I also switched from a pair of TBS tuner cards to using a completely separate Digitbit R1. That broadcasts the four tuners as DVB>IP, which means TVHeadend can use those as tuners over the LAN. TvHeadend also acts as a recorder. TVHeadend just does not have the legacy baggage that Mythtv does. Much less functionality, but much simpler to configure (no need for a Mysql db to store settings and recordings, just a fairly simple HTTPS interface).

The Digitbit’s firmware is quite ancient. but it is trivial to boot for a new firmware via a USB stick (without having to flash the onboard storage) and there are a quite a few forks of that older firmware that support Multistream.

Clients are a number of RPI’s (running OSMC) and a dedicated client from OSMC called Vero 4k. Which is a lovely bit of kit
I also got a Quest Pro II. Facebook blah blah but it’s cheap and standalone. I’m not easily impressed but some of the apps are pretty wonderful. Downside is that our house is quote small and we have so few spaces to walk around in VR! I could use the garden, but then I would look a bit of a tool! Resident Evil 4 in VR is bloody scary, but I’ve yet to acquire my VR legs and can only play for 20 minutes before nausea takes over.

I carried on with the outdoor kitchen and added walls (using rural style corrugated sheets) plus completely weather proof kitchen cabinets (using treated wood and marine ply. All exposed cut edges were soaked in epoxy, and topped with quarry tiles. I need to do the other side. but perhaps when it’s warmer.

In sadder news our dog died. Zorro, our ten year old black lab, started wheezing and it turned out he had a mouth melanoma that was affecting his breathing. Treatment was possible but would have meant removing most of his jaw and chemo for months. Nothing you could put a dog through. Sadly he was put to sleep in the boot of a friend’s car and died in my arms.

I had a pretty bad year for growing stuff. Potatoes, cabbages (in poly tunnel), strawberries and tomatoes (grow bags on the balcony) were great. But everything else did badly I can only blame my laziness in watering during the heatwave. Even our courgettes failed! Better luck next year.

Finally managed to get camping, both for a fortnight in France (super warm), and also at Bluedot festival (brilliant music, shitty weather! Again see below).
I also upgraded our camp kitchen, with a brilliant foldaway kitchen, and a camping stove (Cadac) that has double burners, two griddles, and connects to a gas unit that uses three cheap, ubiquitously available, aerosols gas canisters. It provides a regulator for them too. I’ve always thought they are a little on the dangerous side. But with this bit of kit it makes them very usable and very safe.

We also had a good year for boating. The boat needs a LOT of work doing still. But we had a good few outings including one with the village.

Our own health is decent. LN had Covid over NY (which made that quieter). I’ve still not had it (that I am aware of).
The world continued to become a scarier place. Both nationally and internationally. What is happening in Ukraine is beyond horrendous. However, I decided that ranting at the news is pointless. Whereas I’ll never forgive any unrepentant Leave voters, I guess I’m used to it. Still I had a good day out on the Rejoin March back in August.

It was a good year for live music. Made the Bluedot festival, which had too many decent bands to list (although Yard Act are brilliant!). saw Billy Nomates, Fontaines DC, Scalpings and Working Mens club.

I properly made the move to Mastodon. Twitter was only ever a pleasant social media community in the early years. For some time it’s been both essential to keep up to date with local and national news, but also very unpleasant to be in. Mastodon is some time away from replacing that function, but by being a pleasant place to be in, it works for me. I think Elon Musk has done us all a favour. Back when I joined Twitter (2006) I thought it was a bad move to put all our communication eggs in one company’s basket, but in those early years it seemed to work and before long everybody and their rabbits had an account.

Finally I decided I need to stop doing as much “community stuff” as I have been doing. As well as the village newsletter which I have done for over ten years, I am also a school parent governor and part of a local campaign to prevent a sewage works being built on Greenbelt just outside the village. I find myself spending a hell of a lot of time, but both those latter two tasks have a lot of frustration. One for an incredible lack of communication and the other for manipulative people that are quite incompetent too. If either were paying jobs I’d have left both a while ago.