Guide: How to set up Ubuntu and RAID5 on the MediaSmart Server

by Alex Kuretz on June 7, 2011 · 10 comments

in Guides

The following is a guest article contributed by forum member (and one of my former HP co-workers) “Pete”, who has also shared his LX195 MythTV setup with us in a previous article.

I have successfully setup Ubuntu and Raid5 on a couple of different Media Smart Servers, and found the process not all that bad. While at times Linux setup and configuration is not as nice as Windows, I have found I now have a system that performs better (Raid5 I/O throughput seems much faster than Windows Home Server Drive Extender) and remains stable. Stable and good I/O rates…. those are two key items for network storage….. am I right?

I am sure some people will want RAID10 or some other variant of a setup. The guide here is intended to setup 3 or 4 drives of the same size as a single pool of storage with some data redundancy / protection. You can certainly setup more specific or exotic setups, this guide might be considered a starting point to those really interested in some of the more exotic options. This exercise also likely dedicates too much space for swap, but in the grand scheme of things this should not make much of a difference.

NOTE: The following content has been tested on MediaSmart Servers with 480 and 490 model numbers. The 470 servers contain a different type of Sata controller and I have not got it working yet.

This guide will closely follow other guides already available. Here is another: https://help.ubuntu.com/community/Installation/SoftwareRAID

  1. Obtain the Ubuntu “alternate” install ISO, burn it to a USB key or a USB CD drive. For this example, I am using 11.04 (Natty). These steps may vary a bit from release to release, but largely are the same. The reason we want the alternate install is that it is the main/recommended way to install Ubuntu to a RAID array. http://www.ubuntu.com/download/ubuntu/alternative-download
  2. Turn off machine, get break out board / cabling setup (keyboard / monitor etc). If you have not obtained a console cable for your server, see here. (NOTE: I had some networking issues during the install, so I kept the network unplugged during the install. YMMV)
  3. Grab 3 or 4 (using Raid 5 here) drives of the same size, they will all be completely wiped clean…. so make sure nothing important is on them.
  4. Boot the machine with only the USB device with the OS image on it. Do not plug the drives in 100% just yet. This approach will ensure we boot to the USB device and not one of the drives.
  5. Ubuntu will boot up in an initial screen from the USB key. On natty, this is a language selection screen. As soon as this shows up, plug in your drives. You want to plug them in fairly early…. so the installer knows you have drives to install to. As a rule, make sure the drives are all plugged in before you proceed pass this screen:
  6. After hard drives are plugged in, select the “Install Ubuntu” menu item. Continue selecting the language / keyboard layout, providing host name etc. until you get to the hard drive setup screen. (NOTE: as stated before, I installed without the network cable plugged in, and I had to “continue without configuring network” during one of the screens)
  7. Select manual for the hard drive setup screen. For each drive select the drive itself (not a partition within the drive) and hit enter. You will be prompted “do you want to recreate partition table / remove all data?” Select yes. At the end, you are looking for each drive to contain “FREE SPACE” and not any particulate partition, see below:
  8. After verifying each drive has “FREE SPACE” listed (i.e. no existing partitions), select each “FREE SPACE” item and hit enter, and select “automatically partition”. This will create a swap partition sized around the same as your memory, and the rest of the disk ready for a single partition (likely ext4).

  9. After you have all your “FREE SPACE” items converted into the two items, select the “Configure software RAID” near the top. You will be prompted to save your work above, confirm and proceed.
  10. Select “Create MD device”, and select RAID5. Next type in the number of drives you will use. (If you have 4 drives, technically you could setup raid with 3 drives and setup a hot-spare). On the next screen, you will be prompted with devices you want in the RAID array. Select all the same sized items. For my work, I typically setup the larger partitions first (the first drive is refereed to as md0, second is md1, etc). You might need to use arrow keys/space bar to select the items. Hit continue and save your changes. This screen will vary a bit, but you get the idea:
  11. Follow the step above again, this time for the remaining file system. If you selected all the larger file systems above, you will only select the remaining file systems for swap.
  12. Once you have gone through the raid setup steps twice, select finish. This will bring you back to the disk partition screen used for steps 7 and 8. Now you should see your 2 raid devices. Note that all drives are shown, the RAID devices first as well as the real hard drive devices.
  13. Within the larger raid device, select the line starting with “#1″ and hit enter. This will bring you to a screen where it asks you how to use this partition. Select “use as ext4″, mount at “/”. Select “done” The image below is close to what we want….. except ext4 is likely going to serve us better (pun intended).
  14. Within the smaller raid device, select the #1 item. This will bring you to a screen where it asks you how to use this partition. Select “use as swamp”. Select “done”
  15. Take a breath. Right now, you have done the main work of setting up the raid partitions and are ready to progress. Select “finish partitioning and write changes to disk”. Confirm your changes.
  16. The installer will take off, and run for a while, copying everything to disk. You still a trick or two left, so don’t go too far away just yet.
  17. Setup your default user name and password, don’t encrypt your home directory (it shouldn’t really matter either way)
  18. This can be a tricky part. It will ask you to install the grub boot loader. Select yes to install it. At times, I have had this fail and it tries to install the boot loader to /dev/md0. When this happens, you might need to change the boot loader install path to be “/dev/sda /dev/sdb /dev/sdc” (you might need to include another /dev/sdd also). This tells the grub boot loader to install on your first drive (/dev/sda), second drive (/dev/sdb) and so on. You want all the drives to have the boot loader installed on them.
  19. After grub installs OK, you will reboot and have Ubuntu installed with Raid. Good job!
  20. Charlie Sheen says RAID5 is winning.

More tips

Increasing raid re-build speed
When you first complete your install, the raid array will be in a funny state where it needs to re-build and re-sync everything. Linux just seems to do this from time to time…. re-sync and verify the raid array is all good. This process is slow. I mean really slow. There is a very good article here that helps describe how you can change the raid rebuild speed. I find that a max of around 300000 and a minimum of 60000 works pretty well, and recommend adding these values into the file /etc/sysctl.conf. This can make the difference between a raid rebuild of around 5 hours vs 12 hours. If you want, you could make this value lower…. it is really balancing CPU & IO load vs the time it takes to rebuild the array. If you have a dual core machine, go with this higher value.

Setup auto login
Most of the time the default installer will setup your machine to auto login as your initial user. If not, see here for how to change it. You will likely want to also setup the desktop session to coordinate with the following item “Use a simpler desktop”.

Use a simpler desktop
The new desktop in Ubuntu 11.04 is somewhat of a hardware hog and overkill for a machine where the desktop is not really important. For our server, we should switch back to the default Ubuntu desktop and menu session. See here for more info on how to simplify your life. (I recommend “classic with no effects”)

Adding remote access with Vino (VNC)
Lets be honest…. that console cable stuff we used above? That is not the prettiest thing. If you setup Vino, then you can just use a VNC client to connect into your desktop. Make sure you do NOT select “confirm each access” (which means you will never connect, since you are trying to remove the console cable there is no way to approve) or “configure network automatically” (which will attempt to UPnP program your router to make your machine available on the public Internet).

Also of note for remote access is http://www.webmin.com/ and Ubuntu SSH.

Setting up Samba (windows file sharing)
If you access your Ubuntu box in the Applications > Ubuntu Software Center app, look for an item called “system-config-samba”. This is a small UI to help setup shared folders. Samba can get complex quickly with its own user names, passwords, and permissions. This UI does not really solve all those Samba items, but a good way to get a share setup on the network.

If you want all the bells and whistles of Samba setup, try to install “gadmin-samba”. It is most everything you might want.


Article by

I'm Alex Kuretz, and I'm the founder of MediaSmartServer.net. I was the Lead Test and Integration Engineer at HP for the MediaSmart Server until April 2008 when I moved on to other opportunities outside HP. I've kept active in the Windows Home Server community, creating several add-ins and helping users make the most of their Home Servers.


{ 10 comments }

JohnBick June 7, 2011 at 8:29 am

Interesting article! Well written and informative.

I have been seriously considering Ubuntu as a WHS replacement for one of my clients although I have been looking more toward a new hardware build rather than re-using the existing HP MSS.

WHY? Because we project the capacity of the MSS drives (all 2TB) will be exceeded within a year with the current usage and the client hopes to expand significantly through an acquisition a year from now. Rather than “kludging” an expansion chassis we’re looking at a large frame. And with that we can get hardware RAID, high-end processors, the higher transfer rates and USB3 offered in new motherboards.

Jim Metcalf June 8, 2011 at 2:01 pm

You smell that? Do you smell that?
Money, son. Nothing else in the world smells like that.
;)

What WHS functionality are you looking at replacing with Ubuntu? Are you just going to use Ubuntu for storage?

Pete June 11, 2011 at 7:35 am

I replaced the file shares, media streaming (using mediatomb – so far so good). Windows7 has a built in backup option, though to be honest I do have an existing WHS I keep for client PC backups.

It’s a shame there aren’t better NAS options….

Some are too cheap and not very configurable. They work pretty well for intended use, but you are stuck with intended uses.

Then others like WHS boxes are a bit pricey for extra flexibility. I think something like the proliant micro server (see shollomon’s comments) is a pretty good deal.

Seems to me like a NAS needs to be cheaper than a PC, but every bit as flexible. I think linux helps deliver exactly that.

JohnBick June 12, 2011 at 8:14 pm

That question, Jim, is the real key. My primary use of WHS (and that of ALL my clients) is for backing up client PCs. And WHS is, by a fair margin, the #1 solution for that! The ability to image disks combined with the ability to recover individual files OR disk images based on the date they were archived is really beyond the ability of any other backup package. It really is an ARCHIVE functionality, not just a backup.

Therefore the primary use of the Ubuntu server would be for file storage, but it needs to be in association with a good archival tool. (But I have yet to find one.)

But for the secondary uses (media streaming, sharing data, print serving, web site hosting, general functional serving, etc.) there are Ubuntu solutions, most equal to or better than WHS. In this area there is plenty of competition.

But that primary requirement is a problem…

TedC June 8, 2011 at 7:54 am

Yes – an awesome article for a how to.
I’m also considering re-building the MSS to an ubuntu server. The only things I think I would miss are Orb Transcoding / streaming to my iphone and the ease of use when it comes to setting up your own domain name and web services (The automatic configure of IIS is awesome! in WHS).

I’ve looked for alternatives to Orb and there are a few mentions – but nothing that’s as well established or as easy to use.

As for setting up Apache in place of IIS – well – that’s a whole new “How-to” guide – isn’t it?

One other thing I would mention Alex – I am a serious Ubuntu Hater. I love the OS – don’t get me wrong. I love the idea of a piece of “human” software. But Ubuntu really falls short of this “human” concept. Anything I’ve ever tried in it though for a HTPC has been a monumental feat. From HDMI with Audio through XBMC to installing a Hauppage HDPVR box to getting a fire-wire connection to work to my set-top cable box was a nightmare. There’s so much bad advice on the forums as well – that you usually do more harm than good when following someone’s suggestions. I hate to say it – but when it comes to “out-of-the-box” experience – Microsoft has it made.

shollomon June 9, 2011 at 9:12 am

Setting your own domain name is very easy in Linux/Ubuntu. If you don’t have a domain use something like DynDNS, their client and domain names to point to your machine. If you already have a domain switch its DNS from your registrar to something like ZoneEdit and use one line cron job that runs every 5 minutes to update your ip address. Here’s the one liner for ZoneEdit:

wget -O – –http-user=username –http-passwd=password ‘http://dynamic.zoneedit.com/auth/dynamic.html?host=your.hostname.com’

Something that was, I think, impossible to do under WHS v1 is something just north of trivial under Ubuntu.

I run, and recommend, an HP Proliant Microserver as my home server. I use software Raid, 2 250gb hd’s raid 1, two volumes, 1 mounts /, the other swap. 2 2tb hd’s raid 1, one volume mounted /home. I do not run a GUI.

Most admin is run from bash shell (for example ufw, the firewall system is very easy to manage), for more complicated stuff WebMin is easy to use. iPhone management with iSSH, iPhone file access with ezShare.

The advantages are all around SSH and SFTP for remote access. Use something like WinSCP to transfer files, full drag and drop explorer like interface, without using a browser. I find much faster access to files, much easier to use. Its pretty easy to chroot users into their home accounts (if that’s what you want). Streaming music/video comes from minidlna, it does not trancode, but does a good job otherwise. There is a paid version of Twonky that will do that. Dropbox runs just fine on headless Ubuntu. I run scheduled Calm AV scans of shared folders.

Disadvantages are backup. I run Genie Timeline on the windows machines I backup to the server, it gives a Time Machine kind of functionality, works great. The macs I backup to the server use Time Machine which is a pain in the neck to get working the first time, but thereafter, no problem.

My biggest complaint about WHS was remote access. I can’t for the life of me understand why MSFT does not have a secure file transfer protocol imbedded in Windows. I know I could run SMB over VPN but that seems to be a major pain to configure and get working. SSH is lightweight, easy to configure, reasonably secure, platform agnostic, and just works.

Jakub July 19, 2011 at 10:59 am

How would I ‘attach’ a new 4 disk (2tb) RAID5 array to a pre-existing Ubuntu server? I want to have the RAID5 for media only.

I want it to be able to be expandable (add another PCI RAID card later, for up to 8 disks, etc).

Thoughts?

Emmi May 30, 2012 at 1:11 pm

Hi, thanks for the article. Im thinking of the same, installing ubuntu but my Unit is a ex47x. So my question is do you think that it will work on my box??? And what should i go for the server version or the desktop version when installing ubuntu

Wayne December 19, 2012 at 8:09 pm

Hello,

I can’t get passed the auto recovery mode. How do I disable that and boot to USB drive?

-W

Andrew February 26, 2013 at 11:53 am

Thanks for the Article…. I am looking at doing the same thing…. I tried the Windows 2012 and found the system is really loud now that I lost the fan control from the original OS…..

I am also looking a creating an XBMC server with all the normal apps Sick Beard, TV Headend, etc.

My question is do I need to use the Server version and if I do how do I enable the desktop to create shares and things like that?

I have also read on other forums that the 490 has strange network drivers that might create an issue did you have this issue and have you run into the problem?

Sorry new to Ubuntu but can’t handle the noise of the fans and don’t need a windows box to run a couple applications in the background that are all accessed via web interfaces.

Thanks

Comments are closed, visit the forums to continue the discussion.

Previous post:

Next post: