about:benjie

Image from Wikipedia

I moved my blog (in fact my entire website!) over to Wordpress a couple of days ago. The move was not without it’s challenges - for a start I remembered Wordpress likes to have a well defined hostname, and I didn’t want any downtime. To get around this, I placed an entry in my /etc/hosts file for www.benjiegillam.com, pointing to the new domain, this way I could set up the new Wordpress blog privately (no-one else would know where it was) under the correct domain name, whilst still having access to the old blog to copy content over from.

My first issue was how to transfer the posts from the old blog to the new. I acheived this by doing a few minor hacks to serendipity, and using the export function (where you can export all posts as an RSS feed). To do this, I had to disable the “extended body” feature (i.e. make sure it was output as part of the feed), as explained in solution, part 1, here. Make sure your browser is not caching at this stage!

Once I had acquired the RSS file, I then had to convert it into a format that wordpress would understand. I cheated and wrote a very bad PHP file, here:

<?php //Import the feed $rss = file_get_contents('s9y.rss'); //Opening <![CDATA[s $rss = str_replace("<content:encoded>","<content:encoded><![CDATA[",$rss); //Closing ]]>s $rss = str_replace("</content:encoded>","]]></content:encoded>",$rss); //Now replace all newline characters with a " " (this will BREAK any preformatted tags, but will stop wordpress putting <br />s everywhere $rss = str_replace(array("\n","\r")," ",$rss); //Finally remove all the htmlentities from the file and output to STDOUT, which you can then redirect to a file echo html_entity_decode($rss,ENT_COMPAT,'UTF-8'); // I called this as convert_s9y_rss.php > wordpress.rss Download this code: /code/convert_s9y_rss.phps

I then used Wordpress' RSS importer to import the posts (no comments, unfortunately). I then copied all of the uploaded files into the same file structure on the new site. The next thing to do was to go back through and edit all the posts and update their links. Only joking, I really couldn't be bothered to do that! Instead, I made a folder called "serendipity" in the webroot (all of my posts were /serendipity/archives/... previously), and placed in it the following two files:

RewriteEngine On #Direct *EVERYTHING* to the index.php file RewriteRule .* index.php [L] Download this code: /code/s9y_htaccess

<?php //What URI was I accessed as? $uri = $_SERVER['REQUEST_URI']; //Remove everything except the last section $uri = explode("/",$uri); $uri = array_pop($uri); //Convert to lower case (as in Wordpress) $uri = strtolower($uri); //Remove the post id from the beginning of the post $uri = explode("-",$uri); array_shift($uri); $uri = implode("-",$uri); //Remove the extension (.html) $uri = explode(".",$uri); array_pop($uri); $uri = implode(".",$uri); // Now send a 301 Moved Permanently and the new location header("Location: /$uri",TRUE,301); exit(); Download this code: /code/s9y_index.phps

These caused all posts links to be re-written to a guess at the page name, and thankfully Wordpress was clever enough to work out what was meant. I am not sure if it worked for all posts, but it did for all that I tested.

I hope this helps someone, if so leave me a comment (please! I lost all my old comments in the move!).

Image from Wikipedia

On 5th February, only of my hard drives failed, and my computer started to choke. My root partition was on the failed drive, so only programs that were already in RAM could continue to run, though most of my storage was on a RAID-5 across 4 320GB disks, so that was still intact. Fortunately, due to my 6GB of RAM, swap was not in use also, so this did not crash my PC, but it did leave me in quite a bad position, not being able to run many of the built in system tools, in particular the tools from “smartmontools”. I did what I could to copy the important details over to Jem’s PC whilst my system was still running, albeit crippled.

It turned out that, upon rebooting of the system, everything was fine, and the disk worked, but it didn’t half give me the willies! I vowed then to get my root drive onto some form of redundant storage, and to have a hot spare always on hand. To that end I bought an Icy Dock 5-in-3 SATA cage and 2 500GB SATAII drives. Unfortunately, due to the amount of time I was spending working on buzzspotr.com with i-together, I was unable to actually use these immediately. I finally got round to incorporating them into my system last weekend. It was quite a challenge to do, so I thought I would document it for future reference.

The first thing I did was delete as much data as I could. The main things I focussed on were:

• Old MythTV programs that I had recorded and seen, or that I was not going to watch
• My ripped DVD library (I rip my DVDs to make it easy to get them to play from MythTV without having to get out of the sofa! I could always re-rip them later)
• Old duplicate backups (for example, I had backups every 30 minutes for Blog Friends, which summed to almost 50 GB! I removed all of these except for those from 11:30pm each night)
• Duplicate files
• Caches

After removing all this data I reduced the “valuable” data on my computer to somewhere around 650GB.

I decided that the best way to lay out my computer data would be to have the following:

• First 0.5GB of each drive - swap space and /boot partition. I chose to not make the swap redundant as it is rarely used (and I don’t mind if computer crashes if it HAS to!)
• Next 39.5GB of each drive - RAID1+0 for /, totalling 117GB of fast redundant storage (theoretical peak bandwidth: 1800mbps read, 900mbps write). High priority data here - the root, my home directory and desktop, the mysql databases, the webroots of apache, etc etc. Ultimately everything where speed and redundancy are highest priority. This setup allows the loss of any one, and possibly up to 3, drives, and has less probability of a total fail than RAID0+1. I did *NOT* use the kernel raid10 driver.
• Then 80GB stripes over 5 disks of RAID5, with one hot spare (on the 6th disk) up to the capacity of the 320GB drives, which would all be combined through LVM into one huge partition for lower priority data - music/TV/etc. Note that RAID5 write speed is not great.
• The rest of the 500GB drives are currently unpartitioned, but I might use them as overflow for MythTV, or as hot spares for the RAID1+0

This seemed to me to be the best way to lay out my filesystem, but how on earth could I move my current data over to the new system, and be assured that it would still boot?

My previous setup was (4 320GB drives, remember):

• First 0.5GB swap and /boot
• Then 30GB stripes for RAID5, up to the last 20GB. Combined with LVM, two logical volumes.
• Last 20GB was used for /home on one drive, 64-bit / on another, and a 32-bit / on another. The final drive was blank.

My first issue was how to boot into the Ubuntu LiveCD, and still have volume management. I found the best way to do this was the following:

1. Boot the Gutsy Gibbon LiveCD, remembering to set screen resolution and keymap (for some reason, it crashes for me if I don’t…)
2. Open up synaptic
3. Edit the software sources (repositories) - tick all the boxes, and all the updates boxes (gparted is broken on the LiveCD if you try and use it on a completely raw (fresh from manufacturer) drive)
4. Install all the updates (you can leave out obvious things like OpenOffice.org if you want)
5. Install mdadm and lvm2 packages
6. (Optionally?) Run modprobe raid0; modprobe raid1; modprobe raid5
7. Then run mdadm -A -s –no-degraded
8. (Optionally?) Run modprobe dm-mod
9. Run vgchange -ay
10. Now you should have all your RAID and LVM partitions up and running

If you don’t understand the commands, I highly recommend that you read their man pages to ensure that these are the right commands for you. You CAN lose data if you mess this up! I always have to check each command 4 or 5 times before I run it when this much data is involved!

Once I had done that, I had to take the plunge. First I checked each of the NEW UNFORMATTED disks with a long read-write test (actually, I did this before ever rebooting), by running badblocks -s -w /dev/sde. This is a destructive command, please be careful using it! It will erase any data already on the drive.

Once I was convinced that the drives would withstand the 2 days where they would be the critical data point, I partitioned them. Both got the standard 0.5GB and 39.5GB partitions at their fronts, and then sde got 3 80GB partitions, and the rest (220GB) turned into another partition; whilst sdf got the rest (460GB) turned into one large partition. I then copied everything over to these drives (starting with sdf, and then working backwards through the partitions in sde). I then had to take the jump and make my RAID1+0 (which was formed of striping the pairs sda-sdd, sdb-sde, sdc-sdf).

It was at this point I thought I would be clever. If I just deleted the partitions of sda, then the RAID5 would still be holding the data, and I could make my severely cripped RAID1+0 (really only a RAID0 in this idea) by combining sda5, sde5 and sdf5. I could then copy the data over and check if it booted, whilst still having lots of redundancy for my data. Unfortunately, the system would not boot like this (I guessed it was because I had two md0s - one for RAID1+0, and one from the old RAID5, though I was later proven wrong), so I had to give up and take the risk. I deleted the partitions from the other drives, and formed my RAID1+0, and tried to boot into it. It still would not boot. I even chrooted into the new environment and ran update-initramfs and update-grub, but still nothing.

At this point I was a little frustrated, and spent a long time researching. In the end I discovered that the initramfs was not being updated, and it still contained the old /etc/mdadm/mdadm.conf. Upon deleting and regenerating the initramfs, I could boot into the system. I quickly rebooted into the LiveCD and did all the other necessary changes (setting up the RAID5 across the 5 available disks (leaving the last disk with the data on), copying the data from sdf onto the new RAID5, updating the fstab, etc). I then rebooted into the system, and (not surprisingly) had to make some quite considerably changes due to the amount of data I had moved to new, “better”, locations. And finally, just 2 days later(!), I had my ultra fast and acceptably redundant system online.

I’m very glad that I took the time to do this, though I still have not got round to formatting sdf and setting it up as hot spares… it still has most of the old data on as a duplicate copy!

Image from Wikipedia

My Icy Dock 5-in-3 SATAII cage arrived today from Aria. I was quite excited, and wanted to fit it straight away. I shut my computer down, took the case apart and set about fitting it. The first issue I came across was that the drive rails for my Antec P180 case would not fit properly - I had to fit only the bottom to, as fitting all three meant that they were all too close together to squeeze on the P180’s rails. Once I had this sorted, I set about trying to fit the power and data cables. ARGH! :@ There just wasn’t enough space! And to make things worse, the connectors on the back of the cage were the wrong way round for the L-shaped SATA cables (see image) to fit! After over an hours struggling, swearing, plugging, unplugging and general annoyance, I finally got the damned thing into my case. Breathing a sigh of relief, I set about the long task of attaching my drives to the mounting rails of the cage. I then slotted them in, and rebooted.

WOW!

WHAT THE HELL?!

THATS SO NOISY!

They rattled like anything. I quickly turned the machine off, tightened (really really overtightened, if you ask me!) the screws as much as I could, and then slipped the drives back in and powered up. This time the rattle was gone, but it was still very noisy from all the vibrations. The Icy Dock does not have any anti-vibration built in, making it much noisier than I was used to (I normally mount the drives using the rubber gromits that come with the P180 case). The fan was also very noisy, so much so that I had to change the setting on the back to 55oC instead of 45oC for the warning temperature.

Credit where credits due, the cage does keep my drives nice and cool, and it is very helpful to be able to see the individual hard drive status LEDs. It also have the obvious advantage of being able to quickly and easily swap a drive in and out in the event of a disk failure, without all the effort of having to go inside your case (and possibly knock a cable out of another disk drive without realising!).

Generally, though, I would sum up my feelings thus:

I recommend NOT buying an Icy Dock 5-in-3 SATA cage.Was hard to install&my computer is *miles* louder now+resonates.Pretty lights though…

By the way, this is the first post that I have written with Zemanta (my first “zemified” post). I’m quite impressed at its ease of use, and I will continue to use it for that reason. Towards the beginning of writing a post the articles it brings up tend to be quite random though… what does this post have to do with celebrity babies, for example? I really like the gallery feature though, and the links are especially helpful (though I am surprised that “Zemanta” isn’t among the link detection)! Great work, guys!

I like IBM. I have even visited them a couple of times if you look at my homepage. Well, whilst reading LifeHacker, I saw a link to the following article on the IBM website: Whistle while you work to run commands on your computer. Well, the title sounded like fun so I read on a little. They say it may take around an hour to set up, so perhaps I will do it on a weekend. What made me smile was the following excerpt:

"Note that you will have to be running this program in a relatively noise-free environment, so plug in your headphones and make sure your CD drive is spun down. If your laptop’s battery is on fire, try unplugging the smoke detector before running this program."

I might try and stick more subtle humour like this into the stuff I make for Brain Bakery.

A few days ago I was pretty amazed by video’s of Microsoft’s new technology, which they call "Microsoft Surface," see TechCrunch. Here’s what it can do:

Well, thankfully, I have seen the light since then, due to this video, thanks again to TechCrunch:

But I still think this technology is pretty amazing! Can’t wait for Linux people and hardware enthusiasts to make a competing platform for a fraction of the cost… I can dream, can’t I?