Monthly Archives: October 2012

Cross Flashing of IBM ServeRAID M1015 to LSI SAS9211-8i

Posted on by
1

Finally, the RAID card and SATA cables have arrived. The card was back ordered from the retailer I decided to buy from. Now they’re in stock though and can be found here: IBM SERVERAID M1015 6GB SAS/SATA

Why on earth did you buy an OEM card?

Please read my post SATA Expansion Card Selection.

I also scored some neat cables from Ebay. Here in Sweden, one SFF-8087 cable in the standard red SATA color would cost 150SEK ($22) if I bought one at the same time as the RAID card (no extra shipping cost). Two cables of the same type but with black sleeves over silver cables including shipping from Singapore, 100SEK ($15). The choice was simple…

In my prestudy for a suitable controller I found that it was possible to flash certain OEM cards with the original manufacturer’s firmware and BIOS to change the behavior of the card. The IBM ServeRAID is equivalent with a LSI SAS9240 card and it can be flashed into a LSI SAS9211-8i.

All information on how to flash the M1015 card into a LSI SAS9211-8i can be found in this excellent article at ServeTheHome. The content and instructions are updated so I am not going to put them here in case some major changes occur.

As it happens, the process of flashing these cards does not work with any motherboard. The sas2flsh tool refused to work in the following two boards (PAL initialization error)

  • Intel DQ77MK (Q77 chipset, socket 1155, tried PCIe 16x slot)
  • Intel DG965RY (G965 chipset, socket 775, tried PCIe 16x slot)

The following board worked for me:

  • Asus P5QPL-AM (G41 chipset, socket 775, PCIe 16x slot)

For more information and experiences of the flash process, visit this LaptopVideo2Go Forum Post

Here are some additional information that might be useful to an interested flasher

The board is up and running in the ESXi host now and I will run some benchmarks to compare the performance difference between a Datastore image, a Raw Device Mapped (RDM) drive and a drive connected to the LSI controller and passed through to the VM.

SATA Expansion Card Selection

Posted on by
Reply

In my previous article, The Storage Challenge, I explained my thinking when it comes to storage for the virtual environment. The conclusion was that I need some kind of SATA controller to set up the storage how I want.

 The requirements I have on the controller are as follows

  • Sata III / 6G support
  • Support for 3+TB drives
  • 8 SATA ports
  • RAID 1
  • Drive hotswap

 Let’s break them down.

  • SATA III/6G is not really a needed today as I will start off with only mechanical SATA II/3G drives. However, I might add an SSD drive to the mix later on which would make use of the added bandwidth.
  • 3+TB drives. Once again, I will begin with only 1TB drives but since the main focus of this storage solution is storage and backup I will most likely increase the amount of storage at a later stage.
  • 8 SATA ports. The initial plan is to connect 4 drives to the controller so 8 is pretty much the next step.
  • RAID 1 is not really needed as I plan on beginning with software RAID1. This option is more of an educational decision. RAID 0 or RAID 1 does not require that much from the controllers and they doesn’t get that expensive. Stepping up to RAID 5 or 6 and we’re talking about a completely different price point.
  • Drive hot swap. The goal of this all-in-one machine is that it will always be online. Hence, I would like to be able to add a hot swap HDD bay down the road. I’m also interested in trying how the OS will handle hot swap when the drives are in a software RAID array. Another educational aspect.

With the requirement list set I started searching for a suitable card. Since I run ESXi on the hardware I would like to have native support for it in case I decide to use RAID below ESXi. Looking at the VMware white list for storage adapters (link), the safest bet is to use a card based on an LSI controller. As I have come to understand, many large hardware companies rebrand LSI card to their own brand. These cards happens to be cheaper than LSI’s own cards. ServeTheHome have some excellent articles and summaries on what cards from different manufacturers are based on. Here in Sweden it seems like IBM and their ServeRAID cards are the cheapest with comparable controllers.

 Using the excellent sorting mechanism of Prisjakt.nu boiled it all down to these four cards:

  • IBM ServeRAID BR10i
  • IBM ServeRAID M1015
  • IBM ServeRAID M1115
  • IBM ServeRAID M5110

Here is a great summary on the IBM website regarding the ServeRAID adapters. BR10i is not capable of SATA 6G or 3TB drives so it’s out of the list. M5110 is a nice card capable of RAID5 but is also somewhat 50% more expensive than the M1X15 cards. M1015 seems to be a really popular card to flash to LSI’s own firmwares to enable different operating modes. M1115 seems to be pretty much the same card but I have not yet found any information on flashing it with LSI firmwares. M1015 is slightly cheaper so I decided to go for it.

ServeTheHome also happens to have a pointer to a pretty good deal for people in the United States on a IBM ServeRAID M1015

The Storage Challenge

Posted on by
Reply

I’m having some challenges in deciding how to implement my storage. Of course, I could just go to the VMware Whitelist and pick the latest and greatest RAID controller. The same card would cost more than the rest of the system all together and that’s simply not an option. The goal is to get the minimal card to solve my current need and therefor I need to get a good overview of the reasonable solutions.

  1. No RAID what-so-ever
    This method would cost nothing and be quite simple, hardware wise, to implement – Just plug in the drives to the Intel onboard SATA controller. I could create redundancy with some software routines and set up virtual disk images in each datastore. However, if any disk would go down I would have to shut down the entire system and fix the problem.
  2. Pass-through of onboard controller
    This is an interesting alternative. I could pass the entire onboard SATA controller to a single virtual machine. After that I could do software RAID in Linux on some of the drives. However, it’s not possible to just pass a subset of the SATA ports to a VM. It’s all or none ports. Therefor I won’t have any place to put the VM itself. Catch 22…
  3. Hardware RAID
    Intel’s onboard SATA controller is only capable of software RAID which is not supported by ESXi so this alternative needs a competent controller. There are also some options within this alternative.

    • RAID of the entire environment
      The ESXi would only see the storage in whatever way I set up the RAID. This way everything regarding the virtual machines that reside on a redundant drive set would be covered. This is somewhat tempting but I’m not really sure on how to supervise the RAID sets.
    • Pass on the controller to a VM
      Once again, I could pass the entire controller to a VM and then set up RAID at a software level. The positive thing with passing on a separate controller is that I still have the onboard controller to store non important stuff as ISO images and test VMs. Another appealing thing is that the content of the drives would be only the relevant stuff. The OS resides on the datastore connected to the onboard storage and only the important information would be RAIDed.

I have pretty much decided on going with the last option. Perhaps this is one way to illustrate how i intend to solve it:

  • USB Controller
    • USB Drive with the ESXi host
  • Onboard SATA controller
    • 1TB drive with VMs and ISOs
  • PCI-Express SATA controller, passed on to a VM
    • 4x1TB drives setup with software RAID

Installing ESXi 5.1 to a USB drive

Posted on by
Reply

I do not have any plans to install an optical drive in this server but I still need to install ESXi from an ISO image to a USB drive. This is one way to do it:

  1. Fire up your favorite virtual environment that can handle the following requirements
    • Assign 2 processor cores to a VM
    • Assign 3GB of virtual RAM to the VM
    • Assign an ESXi supported network controller to the VM
    • Enable hardware virtualization to the VM

    Here are some screen shots from VirtualBox running under Ubuntu 12.04:



    Load the ESXi ISO image to the virtual CD drive

    An overview of all settings

  2. Insert the target USB drive into the host
  3. Launch the VM and press a key to halt the ESXi installer at the following screen

    Right click in the USB icon next to the CDROM icon and select the USB drive.
  4. Finalize the ESXi installation and shutdown the virtual machine.
  5. Move the USB drive to the physical machine and verify that it boots.
    Note: The physical machine must be able to boot off a drive with a GPT. Look for a UEFI Boot settings in BIOS.

Issue #1: Intel DQ77MK and booting from USB

Posted on by
Reply

The first challenge I’ve come across is getting the DQ77MK motherboard to boot to a USB device. I want to have ESXi installed on a USB drive and have drives for datastore only. This has given me some nice flexibility previously.

Removing all but a single USB drive with ESXi installed on and setting the boot order in BIOS to this specific device should in my opinion work. It does not. Playing around with boot order, boot up delays (in case the USB drive takes a long time to initiate,) and other boot options such as Boot USB first, boot removable drives etc. does not help either. However, if I boot and press F10 to bring up the boot selection menu and selecting the correct device it works.

I will submit a question to Intel support. I’m actually somewhat interested in how they handle their support since I have not had the need to try it before.

* UPDATE *
I found a solution to the issue. ESXi uses GPT and “UEFI Boot” needs to be enabled to get it to work. This thread over at the Intel Community solved it for me: DQ77MK does not boot automatically from SSD drive.

New virtualization hardware in the house

Posted on by
2

I launch this blog now that I got some new hardware.

The idea is to step up the game a notch or two from the previous system. Here is an article I wrote for NordicHardware about my previous system, the one that will be upgraded now: Vi bygger lågeffektsserver med virtualiseringsteknik

The article is in Swedish but the main point of the article was to demonstrate how to build a low power ESXi server for home usage. The specifications of the system is:

A really cheap system consuming between 45W och 70W depending on load.

The next build will be slightly more powerful and the following components have arrived so far:

PSU is still up in the air as I’m trying to find a solid 80+ gold unit for decent money. Case will most likely be a sound dampened Fractal Design Define Mini. I will dedicate another post to some of the motivation for this hardware and also how i intend to solve the storage part. Component and storage selection are restricted by several factors in this build, which I also intend to cover in that post.