Install Illumian on ESXi 5.0 for All-in-One ZFS Installation

9

One topic I get a lot of questions about is creating an All-in-One ESXi 5.0 machine with ZFS storage based on Illumian, Nexenta, and Solaris 11 Express all alongside guest operating systems. For those not familiar with the concept, the idea is simple, utilize what is arguably the best open storage subsystem very similar to what Oracle and NetApp use in their high-end storage inside a virtual machine. Then, export that storage to other virtual machines running the services you want such as a Windows Home Server 2011 or Linux based LAMP setup. Since the interconnects between virtual machines are fast, assuming you have sufficient CPU to drive the traffic, this method both utilizes underutilized hardware in the ESXi machine while at the same time saving costs on high-performance networking gear. With that in mind, I am going to embark on doing a series of articles walking through the basic setup of the storage subsystem. Today, we are going to get Illumian, an Illumos distribution that serves as the basis for the Nexenta family of products. Of note, Illumian itself is not overly user-friendly and one does not get the slick Nexenta interface, plugins or support, but it does provide a good foundation for those looking to play with an All-in-One ZFS test machine.

Test Configuration

Again it is time to fire up my ESXi 5.0 test bed. If you want to read about the inexpensive, yet very powerful build, you can learn about the AMD Opteron 6128 ESXi server here.

  1. CPUs: 2x AMD Opteron 6128
  2. Motherboard: Supermicro H8DG6-F
  3. Memory: 8x 4GB Kingston unbuffered ECC 1333MHz DIMMs
  4. SSD: OCZ Vertex 3 120GB
  5. Power Supply: Corsair AX650 650w 80 Plus Gold
  6. Chassis: Norco RPC-450B
  7. Cooling: 2x Dynatron A1 G34 Coolers
  8. VMware ESXi 5.0
This server has several LSI cards in it to help test various configurations at this point. Most likely we will be using ESXi VMDirectpath to pass-through controllers to the ZFS virtual machines as that is a leading practice in this setup.

Getting Illumian

The first step in this process is downloading the Illumian ISO ready for the virtual machine. Go to http://illumian.org/ and there is a link to download the ISO. Once you have the ISO downloaded, the next step is to get it onto the ESXi 5.0 machine. I use the following method just because I like keeping active ISOs there in the event I need them. First, select the data store on the server for the ISO.

ESXi 5 Select Data Store
ESXi 5 Select Data Store

Then upload the ISO to the VMware ESXi 5.0 datastore. This is a very simple process and the benefit is that if you ever have to re-install the server, everything will be in one place.

ESXi 5 Upload ISO
ESXi 5 Upload ISO

Prepare the Virtual Machine

The next step is to create a virtual machine for the VM. I already have a guide, albeit with ESXi 4.1 on how to create a Virtual Machine for NexentaStore. Follow that guide and you should be in good shape but some minor differences may present themselves such as now having the ability to use a version 8 ESXi virtual machine.

VMware ESXi Add Memory to VM - VM Properties Memory
VMware ESXi Add Memory to VM – VM Properties Memory

The next step is to mount the Illumian ISO by clicking the wrench and optical disk icon then navigating to the ISO you stored previously on the VMware ESXi server.

ESXi 5 Mount ISO Local Datastore
ESXi 5 Mount ISO Local Datastore

Once this is done, you can choose the Power On command pictured above. One thing I would highly suggest is to first hit Edit Settings and go to Options and change the Boot Options to Boot to BIOS. This will set the virtual machine to boot to BIOS so you can do minor tweaks before everything is setup. One thing that I generally do is move down the PXE network boot to the lowest priority since this is going to be my storage virtual machine.

ESXi 5 BIOS Boot Options
ESXi 5 BIOS Boot Options

Once this is done, hit F10 in the console and Save and Exit. The virtual machine will reboot, and you will see Illumian’s installer. A tip here is to skip installing the 32-bit version.

ESXi 5 Illumian Installer Menu
ESXi 5 Illumian Installer Menu

Now that this is done, you are ready to install Illumian. After I completed the installation, I decided that the installation file was going to be huge. As I am re-launching the ServeTheHome.com YouTube channel in the next week or so, I decided to just make that a quick video. If you are anxious to create an All-in-One ZFS server with Illumian and VMware ESXi 5.0, then good news, the installation is very simple.

9 COMMENTS

  1. Good performance. I think OpenIndiana supports VMXNET3. New NexentaCE 4 might. Patrick heard from those guys on that? Or an estimated release date?

  2. Hey, Patrick:
    Nice website, and great articles.

    Just to let you know from experience, of all the illumos variants, OmniOS is the best. Stable, optionally supported. Unlike OpenIndiana and illumian.

    OmniOS+Nappit rocks. No 18TB limits, no limits on licensing. Only thing missing is RSF-1 clustering. But I doubt home users will need RSF-1.

  3. Does anyone have experience with All-in-One installations using Nas4Free? There is a new version based on FreeBSD 9.2 with out of the box support for VMXNET3 published just yesterday!

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.