One thing I have been looking at for a long time has been doing formal “awards” or best-buy ratings on things I review. What a lot of even regular readers do not know is that I probably only post reviews of about 1/3 of the product I look at. Frankly, if I would rate something as an overpriced 2 out of 10, I would rather look at things that are a better fit for a segment.
Now, the downside is that I generally think that a lot of implementations of ratings or award systems are done fairly poorly. I think we all know why this happens, but it is something I have gone through pains to avoid. At one point I tried to make the process as objective as possible and had gotten up to two dozen dimensions and what amounted to a three-slide “summary” of a review dissecting any aspect I could think of, customized for each type of reviewed product. Clearly, this was not going to be tenable long-term. What I settled on was a few simple dimensions which I could customized depending on what segment a product was aimed at as there are clear distinctions between what one would expect of a system around just to playback HD content versus one to store vast quantities of media.
Design & Aesthetics
Addressing the obvious, aesthetics are increasingly important for electronic devices. Case and point, Apple figured out the design and aesthetics part of consumer electronics and has profited handsomely as a result. Since STH does a lot of pieces on server hardware that often sits tucked away in a datacenter or in a closet, one could argue that this is not a super important item for STH rating criteria. Let me provide two counterpoints. First, big name tech companies spend tons of money making good-looking (and sometimes not-so-good-looking) bezels for servers. Also, there are design features that are functional like an internal USB header on a server board. Therefore, I decided that these areas need their own criteria.
Performance for the Application
Performance is actually very relative. Since this site has been around since 2009 performance has changed drastically. When the site launched Intel had just moved to a 45nm process and a four core, eight thread chip was considered top of the line. Now, I have in the lab 32nm chips that run eight cores and sixteen threads and other 22nm chips that have onboard GPUs and sip power. Performance will have to be relative to the time the review is posted.
Another aspect as perspective on relative performance is that there are tons of processors out these days, and they are becoming more specialized. If one wants to run a home SAP or Oracle instance or a monster ESXi/ Hyper-V server for testing purposes, maybe a 2P server makes a lot of sense. If one just wants an endpoint to watch 3D movies on a TV at low power, then there is another set of performance criteria.
Feature Set
Features oftentimes go hand-in-hand with value. One example that I see often is the inclusion of at least two Intel 82574L or better NICs on motherboards. They are probably the best supported NICs out there at this point and are ubiquitous on server-grade hardware. On the flip side, manufacturers often spend a few more dollars on the bill of materials to add these on boards which after OEM margin, distributor margin and reseller margin often translates into a mandatory price jump. Some applications demand certain features and so this area is going to look primarily at a feature set for a given application. I will strive to ensure that I call out suggested applications in each review to limit how subjective this is.
Value for the Application
This is perhaps one of the trickiest areas to gauge because people look at money very differently. At work, oftentimes projects involve the spending of $100 million or more making a $10 difference trivial. On the other hand, I have been installing my own door knobs, faucets and etc. on the house I just bought because spending a few dollars to have someone else do the work seemed like an excessive cost. Value is going to have to be tied to what the application is. Clearly one cannot expect a multi-socket server board to cost as much as a super low-end board, so by tying the value to the application, the goal is to provide an intelligent frame of reference to evaluate value.
Summing it Up at the End of a Review
After results are tallied, the plan is to provide a quick representation of those results, along with, possibly a recommendation if warranted for a given application. You will start seeing tables highlighting the above and mini summaries in reviews shortly. In the end, I decided to keep this simple. As a consultant, it went through many iterations over the past year or so and I finally found something that I think is workable for now. Stay tuned!
I decided to give some insight into this process so that it does not become a black box where it seems like everything is a best-buy for completely random reasons. If you have any feedback, please feel free to share in the STH suggestion forum.
