SATA Expansion Card Selection

In my previous article, The Storage Challenge, I explained my thinking when it comes to storage for the virtual environment. The conclusion was that I need some kind of SATA controller to set up the storage how I want.

 The requirements I have on the controller are as follows

  • Sata III / 6G support
  • Support for 3+TB drives
  • 8 SATA ports
  • RAID 1
  • Drive hotswap

 Let’s break them down.

  • SATA III/6G is not really a needed today as I will start off with only mechanical SATA II/3G drives. However, I might add an SSD drive to the mix later on which would make use of the added bandwidth.
  • 3+TB drives. Once again, I will begin with only 1TB drives but since the main focus of this storage solution is storage and backup I will most likely increase the amount of storage at a later stage.
  • 8 SATA ports. The initial plan is to connect 4 drives to the controller so 8 is pretty much the next step.
  • RAID 1 is not really needed as I plan on beginning with software RAID1. This option is more of an educational decision. RAID 0 or RAID 1 does not require that much from the controllers and they doesn’t get that expensive. Stepping up to RAID 5 or 6 and we’re talking about a completely different price point.
  • Drive hot swap. The goal of this all-in-one machine is that it will always be online. Hence, I would like to be able to add a hot swap HDD bay down the road. I’m also interested in trying how the OS will handle hot swap when the drives are in a software RAID array. Another educational aspect.

With the requirement list set I started searching for a suitable card. Since I run ESXi on the hardware I would like to have native support for it in case I decide to use RAID below ESXi. Looking at the VMware white list for storage adapters (link), the safest bet is to use a card based on an LSI controller. As I have come to understand, many large hardware companies rebrand LSI card to their own brand. These cards happens to be cheaper than LSI’s own cards. ServeTheHome have some excellent articles and summaries on what cards from different manufacturers are based on. Here in Sweden it seems like IBM and their ServeRAID cards are the cheapest with comparable controllers.

 Using the excellent sorting mechanism of Prisjakt.nu boiled it all down to these four cards:

  • IBM ServeRAID BR10i
  • IBM ServeRAID M1015
  • IBM ServeRAID M1115
  • IBM ServeRAID M5110

Here is a great summary on the IBM website regarding the ServeRAID adapters. BR10i is not capable of SATA 6G or 3TB drives so it’s out of the list. M5110 is a nice card capable of RAID5 but is also somewhat 50% more expensive than the M1X15 cards. M1015 seems to be a really popular card to flash to LSI’s own firmwares to enable different operating modes. M1115 seems to be pretty much the same card but I have not yet found any information on flashing it with LSI firmwares. M1015 is slightly cheaper so I decided to go for it.

ServeTheHome also happens to have a pointer to a pretty good deal for people in the United States on a IBM ServeRAID M1015

The Storage Challenge

I’m having some challenges in deciding how to implement my storage. Of course, I could just go to the VMware Whitelist and pick the latest and greatest RAID controller. The same card would cost more than the rest of the system all together and that’s simply not an option. The goal is to get the minimal card to solve my current need and therefor I need to get a good overview of the reasonable solutions.

  1. No RAID what-so-ever
    This method would cost nothing and be quite simple, hardware wise, to implement – Just plug in the drives to the Intel onboard SATA controller. I could create redundancy with some software routines and set up virtual disk images in each datastore. However, if any disk would go down I would have to shut down the entire system and fix the problem.
  2. Pass-through of onboard controller
    This is an interesting alternative. I could pass the entire onboard SATA controller to a single virtual machine. After that I could do software RAID in Linux on some of the drives. However, it’s not possible to just pass a subset of the SATA ports to a VM. It’s all or none ports. Therefor I won’t have any place to put the VM itself. Catch 22…
  3. Hardware RAID
    Intel’s onboard SATA controller is only capable of software RAID which is not supported by ESXi so this alternative needs a competent controller. There are also some options within this alternative.

    • RAID of the entire environment
      The ESXi would only see the storage in whatever way I set up the RAID. This way everything regarding the virtual machines that reside on a redundant drive set would be covered. This is somewhat tempting but I’m not really sure on how to supervise the RAID sets.
    • Pass on the controller to a VM
      Once again, I could pass the entire controller to a VM and then set up RAID at a software level. The positive thing with passing on a separate controller is that I still have the onboard controller to store non important stuff as ISO images and test VMs. Another appealing thing is that the content of the drives would be only the relevant stuff. The OS resides on the datastore connected to the onboard storage and only the important information would be RAIDed.

I have pretty much decided on going with the last option. Perhaps this is one way to illustrate how i intend to solve it:

  • USB Controller
    • USB Drive with the ESXi host
  • Onboard SATA controller
    • 1TB drive with VMs and ISOs
  • PCI-Express SATA controller, passed on to a VM
    • 4x1TB drives setup with software RAID