MikroTik RDS2216-2XG-4S+4XS-2XQ Power Consumption and Noise
Powered by two excellent Delta 550W power supplies, we would not change a thing about the power setup in this system.
At idle without drives, we were in the 73-77W range. When we had 20x Samsung PM983 960GB drives, we were closer to 130-140W at idle. MikroTik says the maximum power without attachments is 105W. The total maximum is 286W.
In terms of noise we were generally in the 42-47dba range with the system. It is a bit of a bummer that it could not be a sub-38dba system since that would make this great for an office. Still, we were able to record the video with this on set.
Key Lessons Learned
Our Key Lessons Learned fall into two categories. The one that you can see my live reaction to testing on the video and then some areas for improvement. First, the one that is in the video chapter marked “The OH NO Moment with the MikroTik”, and one that reviews that were fast to come out generally missed: This does not seem to support NVMe hot swap! You can see me test this live with Sam in the studio and the realization that the Annapurna Labs processor was not handling the hot swap.
For those who do not know, the NVMe spec was done at a time when the Sandy Bridge Xeon E5 V1 completely changed the market and AMD went into its winter prior to EPYC. As a result, the hot swap, especially hot plug feature of adding a NVMe device back into the system was developed over the years on Intel Xeon E5. Drive makers tested their drives on Xeon and if they worked, they were set since Intel had 95%+ of the market. Then in 2017 the AMD EPYC 7001 came out, and we realized it did not have working NVMe hot-plug at first because while the EPYC 7001 was NVMe compliant, there are bits that go beyond compliance that the industry developed on Intel Xeon E5. That is why in 2018 we had the piece Dell EMC PowerEdge R7415 NVMe Hot Swap AMD EPYC in Action because it took quite some time for folks to develop that feature. You can see in the video the management interface go down every time I removed or inserted a drive.
For storage, that is not what you want. It might be OK for some basic file shares or as a target for IP cameras if you do not mind infrequently missing a few seconds. On the other hand, that is not ideal and not what you would want for virtual machine storage. Hopefully MikroTik can add this in the PCIe switch, but who knows.
That brings us to the 7mm SSDs. I would much rather have 8x or 10x 15mm SSDs instead. If you do not have NVMe hot swap/ hot plug, then you want fewer points of failure, not more. This is 20 drives that can fail and take storage offline. Even if that gets fixed in future firmware, 15mm drives are cheap to get second hand an come in wide arrays of capacities because they are the enterprise standard. We Bought 1347 Used Data Center SSDs to Look at SSD Endurance and for the market this system is targeted at, used enterprise SSDs are perfect.
The Annapurna Labs chip is great. 16 Arm cores at 2.0GHz is great for networking and storage services. Still, I kept wanting to add more memory and do things like run more containers to use extra CPU capacity, but this had soldered memory.
Software wise, MikroTik needs to up its game if it wants to displace Synology and QNAP. If you are accustomed to those interfaces, you get GUIs with contextual help menus that explain what everything does and help guide you through creating RAID arrays and managing shares. MikroTik’s answer is go to the online documentation and then use the CLI. That is not as good.
Likewise, if you want to run containers for Minio or something else, then there needs to be an easy GUI way to do this or most SMB customers will not go that far. Many folks are scared of CLI’s, especially if they are accustomed to using Windows or macOS.
I know this is some hard feedback on the product, but let us face it, if we do not give it, then the product will not improve. This is a product that is so close to doing everything for a SMB that it almost hurts to see it need a little work. Luckily, a lot of this is a software lift.
Final Words
Street pricing for these might end up being in the $1600 or less. With the switch port array and the high-end processor, at that price just as a network switch it would be one of the best priced options on the market assuming you wanted this port configuration. One valid way to view this is that the storage side is just gravy.
Still, this is one of those platforms that I hope MikroTik continues to develop. Having a fast switch, with easy-to-use RAID, sharing via OwnCloud/ NextCloud, VPN capabilities, firewall capabilities, and more can make this an awesome setup for so many SMB deployments. Our 2500 sq ft studio would be a perfect example of this (if we did not run a server/ storage/ and networking review site.) I really like this machine, and I can follow the documentation and get the storage side working. So for power users, there is already a lot here.
This is one where, again, the video has a ton of great stuff:
Meh. Albatross.
One would wish for something with more… configurability and modularity.
Also I don’t see the point of thin 19″ rack if this is to be for small customers with specific needs.
Those typically have bottleneck in direct $$$ not enclosure form-factor and height.
I watched the Raid Owl review of this. He never mentioned the hot swap issue. It’s plain as day in the video you can see it reset. What a fing farce that guy is. That’s why I like real sites not YouTubers that you can’t trust
No hot-swap support with 20 drives is just a no-go. This would be awesome with 4x 3.5″ SAS-3/NVMe -or- 10x 2.5″ 15mm NVMe, if hot-swap was available. The 32GB of RAM is also a hard limit regularly these days, so 128GB would be a much better start.
I think the idea is nice for a small company or branch office, the many different high-speed connectivity is a plus for that price point. But it lacking basic enterprise features makes this more of a homelab piece of kit instead of a good choice for businesses.
This one feels a little odd: as a concept it is super lovable: I just hate the branch-office dilemma of either a whole half-rack of underutilized widgets that could support an office twice to ten times the size but just don’t come any smaller except possibly as awful plastic boxes with wall warts and no redundancy; and it’s got that cheerful Microtik pricing and power consumption that isn’t from used enterprise kit 3 generations old.
However, while its concept is the little do-it-all box the RAM and storage are really, really, inflexible for the purpose: even just a plastics kit option that allows you to replace some of the 7mm support trays with ‘every-other’ 15mm support trays would go a long way; and (given that caches and DVRs are two super common use cases) some option to sacrifice SSDs for bulk SATA(either SATA on the ‘every-other’ 15mm or a variant with 3.5in slots) would be an obvious nice to have; and more or expandable RAM would be super helpful: can you get a lot into really thrifty container configs? Sure. If I am looking to equip a relatively small number of branch locations can I afford a fair amount of RAM for what it would cost to either have more talented people configure my containers or have the existing people put extra time into slimming containers down rather than just grabbing defaults and calling it good? Yes indeed; admin time that you’d want doing that can creep up on you fast.
The RAM seems like just a sticking point(though, as noted, the price is right enough just for the switch side of things that you can forgive some sins; it’s just that you are totally blowing past the point if you run out of RAM and need to drag a separate server into it; since the cost of going from zero extra server to 1 extra server will be much higher than going from 32 to 64 or 128GB of RAM; and adding extra RAM will probably be cheaper on the separate server, at least up to 128, possibly not if you need high density DIMMs); but some of the storage-related complaints seem like they could be addressed depending on what, if anything, Mikrotik or others do with those external PCIe lanes.
Normally external PCIe cardcages seem to be obnoxiously expensive(still often worth it if you can now bring that one PCIe card and just a laptop rather than an entire desktop when travelling; or add a GPU or high speed ethernet to a laptop or the like); but expensive enough that they really ruin the numbers on any “let’s use this mini-PC that has thunderbolt or oculink rather than an a mini-ITX or a short depth server that has a couple of card slots!” project; but if Microtik or someone else pop out a few cheerfully priced and relatively simple options that you can use to add some 3.5in SATA mass storage or just a few arbitrary PCIe cards or similar using those external expansion ports that would add a lot of versatility.
Why don’t they just hire @Patrick to make their hardware. If they want to make a NAS or a server it seems dumb not to just hire him and say “tell us what to do” or at least “hey what do you think of this idea?” I’d dream of Patrick designed boxen.
Well, if they hire @Patrick to build the hardware, who’s actually competent enough to fix the software that’s obviously nowhere near what Synology and QNAP offer?
I was thinking they could add some hybrid cages that could either take four 7mm or two 15mm drives. Have two of the U.2 connectors sit slightly further out for the 15mm option and design some slids that can take either one wide or two narrow caddies.
On a different topic, they could have also used SATA drives. 20 of them would give 6*20=120Gbps, which would more or less saturate the 128Gbps of PCIe bandwidth available. Sure, older tech, but would likely use less power and be close in performance.
My biggest issue that’s stopping me from buying one RIGHT now is that it’s crippled by the slow PCIe speeds and bottlenecks. What’s the point of 100Gb/s connections if you can never approach those speeds if the thing is doing pretty much anything else? Hot swap not being a thing is also a no go as I wanted to toss it into my AI environment as a cache but knowing that one drive failure, no matter what, will bring my entire cluster down misses the entire point.
I can deal with the crappy software as I’m perfectly comfortable in a CLI but it also shows that it’s not a fully developed product I think.
I really want to like this thing but for 2k more I can build something similar with none of those issues