Install OpenIndiana on ESXi 5.0 for All-in-One ZFS Server


Today we are looking at how-to install OpenIndiana on ESXi 5.0 in order to make an All-in-One ZFS server. The basic premise of this method is that one can utilize a virtual machine, passing through SAS controllers with VT-d or IOMMU to create a ZFS-based storage subsystem. Once this is done, it is fairly easy to install operating systems of one’s choosing and make the ZFS storage available to those virtual machines through the server’s high-speed ESXi networking. Another big advantage is that one can consolidate multiple servers into one box instead of having a separate NAS / SAN and virtual machine server. Recently we looked at how to install Illumian, another Illumios derivative, on ESXi for a similar purpose. One of the big advantages of OpenIndiana over the current Illumian version is the ability to use the high-performance VMXNET3 drivers. Let’s take a look at how to get this done quickly.

Test Configuration

Again it is time to fire up my ESXi 5.0 test bed. If you want to read about the inexpensive, yet very powerful build, you can learn about the AMD Opteron 6128 ESXi server here.

  1. CPUs: 2x AMD Opteron 6128
  2. Motherboard: Supermicro H8DG6-F
  3. Memory: 8x 4GB Kingston unbuffered ECC 1333MHz DIMMs
  4. SSD: OCZ Vertex 3 120GB
  5. Power Supply: Corsair AX650 650w 80 Plus Gold
  6. Chassis: Norco RPC-450B
  7. Cooling: 2x Dynatron A1 G34 Coolers
  8. VMware ESXi 5.0
This server has several LSI cards in it to help test various configurations at this point. Most likely we will be using ESXi VMDirectpath to pass-through controllers to the ZFS virtual machines as that is a leading practice in this setup.

Getting OpenIndiana

The first step in to install OpenIndiana is downloading the OpenIndiana ISO ready for the virtual machine. Go to and there is a link to download the ISO. Once you have the ISO downloaded, the next step is to get it onto the ESXi 5.0 machine. I add OS ISOs to the ESXi server just because I like keeping active ISOs there in the event I need them. First, select the data store on the server for the ISO.

ESXi 5 Select Data Store
ESXi 5 Select Data Store

Then upload the ISO to the VMware ESXi 5.0 datastore. This is a very simple process and the benefit is that if you ever have to re-install the server, everything will be in one place. The OpenIndiana server (text) installation ISO weighs in at under 500MB currently, so even on valuable mirrored RAID 1 SSD space this is under $1 of today’s SSD storage price to keep on the server.

Prepare the Virtual Machine to Install OpenIndiana

The next step to Install OpenIndiana on ESXi is to create a virtual machine for the OpenIndiana VM. I already have a guide, albeit with ESXi 4.1 on how to create a Virtual Machine for NexentaStore. Follow that guide and you should be in good shape but some minor differences may present themselves such as now having the ability to use a version 8 ESXi virtual machine.

Select VM Version 8
Select VM Version 8

Another key aspect is to ensure that you are using a Solaris 10 virtual machine type since OpenIndiana is based largely on OpenSolaris.

Select Other Solaris 10
Select Other Solaris 10

The next step once you have created the virtual machine is to mount the OpenIndiana ISO from the local datastore.

Mount ISO Image
Mount ISO Image

Note, I am using the OpenIndiana development ISO here. Once this is done I start the virtual machine and hit F2 to get into the VMware VM BIOS to make a few Boot changes. The big thing here is to ensure that the network controllers are last and the CD-ROM is first in the event that you want to boot later from an ISO, this helps.

ESXi 5 BIOS Boot Options
ESXi 5 BIOS Boot Options

Once this is done, hit F10 in the console and Save and Exit. The virtual machine will reboot, and you will see OpenIndiana’s pre-installation menus. This you will have to select for yourself but is very straightforward.

OpenIndiana Language Select
OpenIndiana Language Select

After navigating through the menus you will come to the OpenIndiana installation screen which looks a bit less colorful than Illumian’s installation screen. A very important thing to keep in mind for the VMware vSphere client console newbies that may be reading this, control + alt in windows will let your mouse escape. OpenIndiana is going to take a minute or two to install and it is a good opportunity to practice that skill.

OpenIndiana Installation Screen
OpenIndiana Installation Screen

Now that this is done, you are ready to install OpenIndiana. After I completed the installation, I decided that the installation file was going to be huge and is fairly easy so, I decided to just make that a quick video. If you are anxious to create an All-in-One ZFS server with OpenIndiana and VMware ESXi 5.0, then good news, the installation is very simple. For those that can wait, in a few days, it will be online.


  1. Hoping you post the videos of the installation soon. Good piece. Like where the coverage is going.

  2. I am looking at a all in one solution myself but have been unable to find any decent (consumer grade hardware) solutions which include performance data over time to warrant a purchase. I am looking forward to seeing what you use this for and what your results look like in IOPS,w/r speeds.

  3. Great write up. I am using Solaris 11 now for ZFS, but want to switch to OpenIndiana, after recently reading more about it. I am sure that you have exported many arrays to import on a new system. Maybe you could write a post about that topic. I will need is when I upgrade.

  4. Great Write up Patrick. I am using Solaris 11 now, but haven’t been able to get patches or updates, so I think for a home system I need to move to OpenIndiana. I have been wondering about exporting my array to go to OpenIndiana from Solaris 11. I can’t afford to loose the data so it would be nice to have them imported on the same disks to OpenIndiana system. Perhaps you could do a write up about that so that those wanting to migrate to OpenIndiana could avoid all the pitfalls.

  5. Thanks for writing this up, I’m really looking forward to the video addition your adding in the near future. Thank for all the hard work!

  6. I tried using Open Indiana together with Napp-it on a VMware ESXi 5 installation with iSCSI connection to a WHS 2011 VM. I used the desktop version of Open Indiana because I didn’t figure out how to change the NIC settings to fixed IP for the server version (Googling that brought up a number of different suggestions, none of which seemed trivial enough to try; now that would be something I would appreciate if you covered it here).

    However, I wasn’t happy with this installation at all – when copying large amounts of data to it (i.e., 2 TB at a time – my old server’s HDs), after an hour of copying or so the connection to ZFS “failed” (I don’t know a better way to describe it – the copy job to the server suddenly ends with an error).

    So I moved back to Nexentastor (on the same HW) which never has any similar issues. So it seems the Open Indiana desktop version isn’t suitable as a server platform?


  7. Nice write-up, Patrick!! You’ve emboldened me for the next big project — Now at the end of January 2013, massive changes on the home server front – the garage office of a retired economist and open source contributor. It’s time to consolidate a rack of vintage Proliant boxes for something much like you are doing here. Theory of Operation throughout = ZFS + ESXi, probably using modified Napp-It approach. [If you haven’t yet, a write up on Napp-It and even ZFS Guru would be worth the time!]

    Parts arriving for an AMD 6348-4P server for current work. For maintenance, looking for something like the 6262-HE in a P2 configuration. Done right, I can replace an entire rach of ancient systems. I’ll build these as DAS with connected storage, iSCSI 1K IP on a good switch used infrequently, and ready for high speed network interconnection when I can bloody well afford it.


Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.