Guide: unRAID Server – Part 2 Installing unRAID

by Jim Metcalf on May 19, 2011 · 18 comments

in Guides

This is part two of a three part series of articles serving as a review and basic install walk-through for the storage server unRAID. It attempts to explain the tradeoffs and install details of the popular storage alternative to Windows Home Server (WHS). This guide is something like “I wish I had known this before I started.”


In part one of this series I introduced you to unRAID and explained my reasons for using it to build my next home server. In part two I explain the hardware requirements, preparation steps, and provide a complete walk-through of the unRAID installation process.

Outline of Install

  • Discussion of hardware and software requirements
  • Prepare and boot the USB flash drive with the unRaid software
  • Verify that the hardware is functioning normally
  • Prepare hard drives with preclear
  • Configure settings in web interface
  • Add disks to the array
  • Format disks and parity
  • Create SMB/CIFS shares and set split level
  • Configure security, optional software, and headless operation

Hardware Requirements
Once you have decided to try unRAID, hardware is the first decision. Lime-Tech (the unRAID vendor) publishes a hardware support list, but most systems beyond the list are supported. Below are some basic guidelines when choosing a system configuration.

CPU: If you are doing a small to medium sized build (~3-12 drives), then any CPU faster than 2Ghz will do fine.

Memory: 2GB is plenty. I got two sticks of 1 GB RAM to take advantage of dual channel on the motherboard.

Motherboard: The best have six SATA ports on the board. This gives you up to 10TB of protected data (using 2TB drives) without having to resort to an additional card. You want at least one PCI Express slot for potential expansion. Also, there are some gotchas with network chipsets, but esp. stay away from any Atheros NICs. Any video output is sufficient.

Hard disk drives: unRAID has recently added support for Advanced Format drives and many unRaid users have used the Western Digital EARS green drives. These are not bad drives, but I made a mistake* with them that required me to rebuild my array. I would purchase the HITACHI Deskstar drives if I were starting from scratch, but any SATA hard drive will work.

Power supply: Most Hard Disks consume less than 15 watts each, so the most important aspect is a single 12V rail that is rated around 400 watts.

In addition to DIY, there are vendors who sell prebuilt unRaid servers:
http://www.greenleaf-technology.com/
http://www.lime-technology.com/products/

After some investigation, I decided on a budget motherboard that was not on the HW compatibility list. Below is my kit and the prices that I paid about a month ago (April 2011). This has been a solid config:

The total cost was $290 – $365 depending on using the free version of unRAID.

Installation Process
What follows is a lean version of the Lime-Tech configuration wiki. It should be enough information and explanation to get a Linux newbie running with two data drives and a parity drive (the maximum for the free version) with the most recent version (4.7 Final) on a typical budget PC.

unRAID is installed on a flash drive that is used to boot the machine. This is a nice feature because the OS does not take up a SATA slot that could be used for a hard drive. Also, if your motherboard fails, you can use the flash drive and your existing hard drives without having to rebuild the entire system. Also, backing up your flash drive config is super easy.

For this install, you will need:

  • An assembled workstation with a keyboard, monitor, and at least two hard drives
  • Internet access
  • A second computer with a web browser
  • USB flash drive
  • Some configuration files listed below

You will need to register at Lime-Tech in order to download some of these files. After the install, the server can be headless (no keyboard or monitor) using a Telnet client, such as Putty.

After you register and login to the Lime-Technology forum, you will need to download these files:
[1]UnRAID Server version 4.7 Final
[2]Preclear script
[3]Unmenu

Before starting the installation, you need to do some housekeeping of your BIOS:

  • Set your BIOS to boot from a USB
  • Set SATA access to AHCI mode
  • Check that your BIOS recognizes all of your drives

Preparing your Flash Device for unRaid OS
To make a bootable USB Flash device using Windows (XP/Vista/Win7), follow these steps:

Attach your USB Flash drive and find ‘My Computer’ (XP) or ‘Computer’ (Vista/Win7) and right-click your Flash device. Click ‘Format…’, set the volume label to UNRAID (this must be in all caps) and click Start..

After the format is complete, click on your USB Flash drive labeled UNRAID and copy the contents of the unRaid Server zip file downloaded from above [1] to the flash drive. There should be a file named “make_bootable” at the root of the USB drive. For Windows XP, double-click on the file ‘make_bootable’. A DOS window will open and run the ‘syslinux’ utility on the Flash. For Windows Vista or Windows 7, right-click on the file ‘make_bootable’ and select ‘Run as administrator’.

From the second link [2], unzip and copy the file “preclear_disk.sh” to the root of your flash drive. This must be used to prepare the disks for use in your server.

Create a folder named unmenu in the root of your flash drive. From the third link [3], unzip the file and copy the file “unmenu_install” into the /unmenu folder on your flash drive.

The finished file structure should look like this:

Right click your UNRAID USB drive and select ‘Eject’. Your Flash device is now ready to boot into unRaid Server OS.

Alternatively, you can purchase a preconfigured flash drive from Lime-Tech. However, the preconfigured flash drive does not include the preclear script [2] or unMenu [3].

First Boot
Now that you have the required files on your flash drive, plug it into a USB port on your server and use it to boot your UnRAID system. When the system has booted you will be greeted with a screen similar to this:

Select unRAID OS.

If all goes well, you should get a login prompt:

Type “root” and press enter (no password) to log into the system.

Verifying Network Connectivity
To verify that your unRaid system is connected to the network, at the prompt type:

ifconfig eth0

The “inet addr:192.168.1.200” represents the IP address (usually given out by a DHCP server).

The IP address after inet addr will be different, but anything that has 192.168.x.x or 10.x.x.x means your unRAID server has received an IP address from a DHCP server and is connected to the network.

Verifying Hard Drives are Detected and confirm Drive mappings
To verify that your hard drives are properly detected, at the prompt type:

dmesg | grep SATA | grep link

(Note the pipes (Shift+backslash) between commands)

You should get a listing of each SATA controller, its link status, and its speed. The output will look similar to:

To see all of your device mappings, at the prompt type

ls -l /dev/sd[a-z]

Note all of your “sdX” mappings (these should be sda & sdb & sdc for just three drives). Later, you will use these /dev/sdX names to reference the disks.

Preparing your drives using Preclear
Lime-Tech highly recommends running a process called “preclear” that stresses a newly installed drive by writing zeros to all sectors. This process takes significant time (usually ~20 hours) and must be done for all drives. For me, this is easily the most frustrating aspect of unRAID because new drive space is not immediately available for use as storage. This means that adding a drive is usually a two-day affair.

There are numerous discussions on the unRAID forums about the best way to set up your drives with respect to Advanced Format (AF) drives. Most of these discussions concern older versions of unRAID and do not apply to a clean install of the most recent (4.7 Final) version of unRAID. You can safely avoid most of these issues by formatting ALL (even non AF) drives with the “MBR: 4k-aligned” setting.

A special note if you are using the Western Digital Green 2TB EARS drives from your WHS or other system. If you have the jumper installed, you need to remove it prior to installing it into your unRAID server.

All of the files on your USB device will now appear under the /boot directory off of the root on your booted unRaid server. The preclear process is run from the USB drive where you copied the utility files [2].

The script will look like this:

./preclear_script.sh -A /dev/sdX

where x is the letter to your drive mapping that you checked above. The drive mappings should be “sda”, “sdb”, and “sdc” if you put in three drives. If you are installing on a new system, you should preclear all the drives at one time.

To begin the preclear, change directory to /boot and type:

cd /boot
./preclear_disk.sh -A /dev/sda

In this example, sda is the device mapping of your first drive.

The preclear script will prompt you for confirmation. It will look something like this:

########################################################################
Device Model:     WDC WD20EARS-00J2GB0
Serial Number:    WD-WCAYYXXXXXXX
Firmware Version: 80.00A80
User Capacity:    2,000,398,934,016 bytes  
Disk /dev/sde: 2000.3 GB, 2000398934016 bytes
…
########################################################################
Are you absolutely sure you want to clear this drive?
(Answer Yes to continue. Capital 'Y', lower case 'es'):
Double check to make sure the Drive model and/or serial number are what you expect. If everything is copacetic type "Yes" and press enter like the box tells you too. When you do, EVERYTHING on the drive will be DELETED.

Now that the first preclear is running, additional preclears can be started simultaneously by opening virtual terminals (vTerms). Press ALT and F2 together to open a new shell (vTerm2) where new commands can be run.

Begin the preclear on the 2nd drive, change directory to /boot and type:

./preclear_disk.sh -A /dev/sdb

In this example, sdb is the device mapping of your second drive.

Open additional vTerms (using ALT+F3 for sdc, ALT+F4 for sdd, …) for additional drives.

As each preclear finishes, there will be a message on the screen that the drive either passed or failed the preclear process. Use Shift-Page to page up and down the report. You can close the respective vTerms by typing “exit” at the Linux prompt. At this point it is recommended that you save your preclear results.

On vTerm 1 (Alt-F1) type:

grep preclear /var/log/syslog | todos >> /boot/preclear_results.txt

This will save the file preclear_results.txt to your USB drive for documentation.

Once all the drives have finished the preclear, they can be added to the array. For this simple set up, we will add two drives as a data disks and one as a parity disk.

A note on preclear: unRAID can perform this without a preclear, but the array will be taken offline during the process (~20 hours or more for a 2TB drive ). To prevent this, you can run the “preclear_disk.sh” script as described above on each new drive before you add it to the array. unRAID will recognize that the drive has zeros written to every sector and will only take the array offline long enough to perform a format (usually a minute or two).

In part three of this series I will cover configuring your new unRAID server, including setting up the storage array and shared folders, as well as securing your server.





Article by

Hello, I'm Jim. I reside in Athens Georgia with my wife and four kids. For a salary and on occasional late nights, I manage technologies which include VMWare, Active Directory, and NetApp. For fun and necessity, I build systems that deliver the largest bang for a given dollar which include unRaid, Dune streamers, and a movie screen made from a counter top.


{ 18 comments }

AWT May 19, 2011 at 7:41 am

You can alos use external enclosures. I use one 5 bay Sans digital enclosure and four 4 bay sans digital enclosures.
I only have two drives in my unRAID server itself. The cache drive and the parity drive.

The external enclosures are of course port multiplier capable so I have a PCIe x4 port multiplier card that handles four of the enclosures and I have one PCIe x1 port multiplier card that has one of the four bay enclosures attached with three drives.
I found that if I used four drives it slowed down the entire array to around 14MB/s when doing the parity check which was too slow for me. So with only three drives in that one enclosure I get around 23MB/s when doing a parity check which is acceptable to me since I use the cache drive.

Jim May 19, 2011 at 8:01 am

Interesting. What PCIe x4 card do you use? This modular approach is certainly better for cooling.

I don’t use a cache drive because I’m just using unRAID for write once and read many and ~30 MB/s with parity is more than enough for me. However, I’m intrigued by setting up the cache drive as a warm spare. Have ever given any thought to this?

Overall, what have been your impressions of unRAID? Can you compare it to WHS, or did you go this route from the start?

AWT May 19, 2011 at 11:30 am

I used the Rosewill RC-218.
http://www.newegg.com/Product/Product.aspx?Item=N82E16816132018

It has two external ports and I got dual port external eSATA to SATA adapter to bring the 2 internal ports to the outside.

For the PCIe x1 port multiplier card I got this card
http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=280429470355

I ultimately decided to use a cache drive since I could quickly change it to the parity if I had issues. But later i found that I also needed it for faster speeds especially when backing up several PCs concurrently.

If I only put 16 drives on my PCIe x4 card( along with three on the other and on a MB sata port) I can get around 30MB/s rates when writing to the drives. But even still that was not fast enough for how i wanted to use it.

The disadvantage of the external cases is that they do use more power. When all my drives are idle I’m using around 155 watts. When all are in use during a parity check it uses around 315 watts. Although typically on the cache drive is active and I have the unRAID set to move the data every eight hours. 12AM, 8AM, and 4PM.

unRAID has been more robust than my MSS WHS. I have 56TB in that which I strictly use for my BD rips now. With the unRAID and using a cache drive I can get several times the speeds I get when writing to my WHS which makes it great when I do backups and images from multiple PCs concurrently.

Bob Saget June 24, 2011 at 8:18 am

What was the mistake you made with the WD Green Drives?
Was it just forgetting to remove the jumper?

Jim Metcalf June 24, 2011 at 9:36 am

I didn’t make a mistake with the actual jumpers. My mistake was failing to change the default formatting from 512 to 4K when I added the drive to the array.

I precleared using Advanced Format (AF) drive switches with no jumpers, but when adding the drive to the array, the default format is 512 even though it was cleared as 4K. The worst thing about this mistake was that it “kinda” worked. Everything was fine on writes, reads, and parity checks, but performance was poor and there was a risk of silent corruption. This required me to remove the EARS from the array and reformat as 4K. Not that big of a deal since I had not written much to the disk, but the worst aspect was the lack of consensus on how to handle various situations that arise with jumpers/formating/reformatting/et al.

Even after I formatted the drive correctly, the performance was not great. Personally, I would not recommend purchasing new EARS drives for an array. Existing drives are OK if you just remove the jumpers and treat any drive as AF from the start.

stab October 7, 2011 at 1:15 am

Ouch… Only just found this guide!
After three days, I’ve finally got my Flash drive to boot and started setting up but haven’t precleared. What happens if you DON’T preclear the drives?

Sounds like I need to stop the parity sync I’ve got going and start this all again huh?

Jim Metcalf October 7, 2011 at 10:48 am

Stab,

What took so long to get your flash drive to boot?

You don’t need to stop the sync, but it would be a nice precaution.

The preclear is outside of the actual unRaid RAID function. The preclear operation is a test that identifies sectors that could not be read and to make sure the drive reads all zeros after completion. It’s more of a stress test and preparation for use in unRAID.

stab October 7, 2011 at 4:30 pm

So its advisable to do, but the drives will work OK if I don’t do it…

(re the flash, I had trouble with incompatible sticks/configs etc)

Jim Metcalf October 7, 2011 at 6:14 pm

That’s correct.

Have not heard of that before (re sticks).

Sorry you had trouble. I really think you will like it once you get it going. Please report back with your experiences.

stab October 7, 2011 at 9:37 pm

Cool. I’ve only got 400minutes left of parity-sync so I’d hate to have to start again.

I’m using a WD20EARX drive. As a noob I understand I don’t need to worry about the jumper thing with this drive. One thing I didn’t do was change the Default partition format to MBR: 4K aligned before I started formatting the drives.
How important is this?
cheers

Jim Metcalf October 8, 2011 at 9:43 am

There is a lot of debate about this. I believe you will be fine if performance is OK. I would finish up the install and then post performance numbers and go from there.

stab October 9, 2011 at 9:11 pm

I just read this article that said
“You can also improve raw performance by deactivating shared folders. Doing so means that every drive in the array is visible independently on the network but is still secure. Read speeds can reach 70 MB/s and write speeds 40 MB/s, which means you’re not far off reaching the physical limits of a Gigabit connection while writing data.”
http://www.digitalversus.com/lime-technology-unraid-p889_8992_473.html
Is that for real? Once I connected to my Tower via SMB on my Mac, I could see Disk1, Disk2, Flash etc. If I only want to use each disk independantly is this a good way to go?

Jim Metcalf October 9, 2011 at 9:38 pm

This claim might be technically true in some circumstances, but it defeats a very attractive feature (pooled storage) for a minimal gain in performance. I average about those speads with green drives and GigE.

There are times when you might want to write or read from one drive, but you should take advantage of User Shares to pool your disks. There are many benefits to this including:
Streamers are much easier to set up
unRaid can handle balancing all of your drives
I’m not sure that “Disk1″ share will always be “Disk1″ in the event you replace/upgrade a drive or move files.

I guess I’m wondering why you would want to use each disk independently?

Jim Metcalf October 9, 2011 at 9:40 pm

Alsok, here are some good tips if you are interested in improving performance:

http://lime-technology.com/wiki/index.php?title=Improving_unRAID_Performance

stab October 12, 2011 at 8:38 pm

Thanks again for the help and advice ;)
I’ve just had my server go down on me twice; specifically, I’m copying 1-3GB mp4s to the share and I get “you’ve lost connection to Tower”.
UnRAID web guy doesn’t work and plugging in a monitor to the Server directly remains blank – all I can do is hard reset and start over.
Any clues on this problem?

Jim from public terminal October 13, 2011 at 6:40 am

Wow. That is horrible news. Can you telnet into the box? I would look at the logs first.

stab October 13, 2011 at 9:10 pm

Yeah, I’m still working out how to get unmenu to work on my mac via the browser and how to generate a syslog.

kevin February 8, 2012 at 12:48 am

STAB. Pre clearing is just that ‘pre’..if you dont do it it will get done at the point you add the drive to the array, you will not be able to use the array whilst it is doing it and it will take longer, with pre_clear, you can open a telnet session, start the pre_clear process, close and forget for 24hrs, i did 4 x 2TB drive is little under 24hrs, once done they were added to the array and rebuilding took place.

Comments are closed, visit the forums to continue the discussion.

Previous post:

Next post: