Home NAS – FreeNAS

At the moment I use a 4 bay Netgear ReadyNAS NV+ v2 with four 2GB disks in their X-RAID2 setup. It’s basically RAID 5 now but it auto expands as you add disks or swap disks for larger capacity.

As there’s been some clearing out of old kit at work I thought I’d build my own NAS based on FreeNAS and ZFS. Initially I’d salvaged some 3TB drives, 4GB registered ECC DDR3 DIMMs and a Xeon LC3528.

I have an old Antec 900 case, BeQuiet Pure power 700W PSU and two Marvel based SATA cards spare so I thought all I needed was a mobo.

It seems though the the combination of processor socket: LGA1366, and registered ECC RAM makes it nearly impossible to find a second hand motherboard, especially in for form factor I need.

In the end I bought a second hand but ‘new’ Supermicro H8SCM Motherboard & Opteron 4280, sweet! That’s a cost I hadn’t planned on spending so I’m going to have to sell my current NAS once I’ve set my FreeNAS box. As another bonus I was able to get hold of some more old server RAM, probably from our ESX servers, so I’m up to 64GB.

I’ve played with FreeNAS before, same case and SATA cards but with an old Q6600, 4GB of non-ECC DDR2, nine 1TB drives (the dreaded SMOO drives) and a Hiper 580W PSU. It ran fine in the cupboard under the stairs with my now defunct crypto miner but before I cut over to it permanently it started generating a very worrying burning smell!

I tried to use it again in my new build but the smell returned. After opening up the PSU the issue seemed to be copious amounts of dust rather than over loading. I’ve cleaned it out however I decided to migrate to the BeQuiet PSU.

After the initial build I realised any hard disk failures would be a nightmare to replace, so I bought three hot swap 4 drive enclosures, I could now fit 12 drives in my case. Each enclosure had four SATA data, power and 4 pin fan connectors and unfortunately four power leads that were MOLEX to SATA and fan. My PSU had loads of SATA power connectors but only a couple of MOLEX.

I bought some molex to 4 pin fan power at first cables but didn’t wait for them to arrive as I thought they were the wrong connectors; instead I used the power leads that came with it and bought three one to 4 MOLEX splitter cables and one SATA to MOLEX converter. MOAR cables!

I spread the disks evenly across the two SATA cards and the onboard connections, four drives in each.

Now the hardware was set up the fun part started! It seems, to cut a very long and frustrating story short, that I’d hit some sort of device limit of the SuperMicro BIOS.

All was working fine until I stupidly changed the boot order of the hard disks in the BIOS, then it wouldn’t boot from USB devices! It would boot from an external USB DVD drive though. After days of trouble shooting (resetting CMOS etc) if I disabled the onboard SATA and IDE controllers it would boot from USB however I lost 4 drives.

Thinking this was the issue I ordered another SATA card plugged it in and moved the four drives it it. Plugged it all in (also rearranging all the data connections so hopefully the drives are ordered top to bottom) and the issue returned, AAARRRRGGGGHHHHH! It was at this point that I started venting my frustrations at SuperMicro UK’s twitter account, that’s what’s twitter is for right?

I then unplugged one drive and then I could boot from USB. So the repeatable issue was over 11 SATA devices and the BIOS couldn’t boot from USB even though it could see the devices.

I then installed a spare 40GB SSD into an onboard port and installed FreeNAS from a CDROM to it. So 13 SATA devices is ok but I can’t boot from USB :-/ At least I now have 21TB of RAIDZ2 storage for home! Now what am I going to fill it with?

 

 

One thought on “Home NAS – FreeNAS”

Leave a Reply

Your email address will not be published. Required fields are marked *