MikroTik RDS2216-2XG-4S+4XS-2XQ Internal Overview
There is a lot going on in here.
First, on the NVMe backplane, we get a PCIe switch. This again takes a PCIe Gen3 x16 link from the Annapurna Labs CPU and breaks out to 20x PCIe Gen3 x2 links for the front drives.
There are ten 1U fan modules across the middle of the chassis to keep air moving.
These fans use standard 4-pin power connections, which is great.
On the left side of the photo below, you can see the two M.2 SATA II (3Gbps) ports. We put cheap SSDs in there to give us 22 SSDs in this system.
The more impactful features are the two heatsinks that cover the Annapurna Labs AL73400 16-core Arm CPU and the Marvell 98DX4310 switch chip.
The Annapurna Labs AL73400 is a 16-core 2.0GHz Arm CPU, which is quite strong for a networking and storage platform like this.
Next to that, we have ten DRAM packages in sets of five. This is only 32GB of memory, and it is soldered. That means, this is only a 32GB platform. We wish there was DIMM/ SODIMM memory expansion instead. That usually costs more to implement, but if you want to really have a server, then 64GB or 128GB would be nice to have.
Since there are ten packages 5+5, our sense is that this is two channels of 16GB, and likely ECC. We could not test the ECC feature, but usually that is why you see 5 or 9 DIMM packages instead of 4 or 8 in a channel.
The smaller heatsink covers the Marvell 98DX4310 switch chip. This is a chip that is used in switches like the MikroTik CRS504-4XQ-IN.
Here is the quick spec sheet on that switch chip:
The two 10Gbase-T ports require a PHY that has its own heatsink.
The other ports are just pluggable module cages. One difference is that the two SFF-8644 ports you can see the traces that lead to the Annapurna Labs CPU not the Marvell switch chip.
Here is where the redunant Delta power supplies slot in.
Here is the power distribution board.
A small but fun feature you can see is that the cable connector powering the NVMe backplane sticks out a bit, so one of the fans has to be offset slightly to accomodate that connector.
Next, let us get to the block diagram, performance, and the software.
Meh. Albatross.
One would wish for something with more… configurability and modularity.
Also I don’t see the point of thin 19″ rack if this is to be for small customers with specific needs.
Those typically have bottleneck in direct $$$ not enclosure form-factor and height.
I watched the Raid Owl review of this. He never mentioned the hot swap issue. It’s plain as day in the video you can see it reset. What a fing farce that guy is. That’s why I like real sites not YouTubers that you can’t trust
No hot-swap support with 20 drives is just a no-go. This would be awesome with 4x 3.5″ SAS-3/NVMe -or- 10x 2.5″ 15mm NVMe, if hot-swap was available. The 32GB of RAM is also a hard limit regularly these days, so 128GB would be a much better start.
I think the idea is nice for a small company or branch office, the many different high-speed connectivity is a plus for that price point. But it lacking basic enterprise features makes this more of a homelab piece of kit instead of a good choice for businesses.
This one feels a little odd: as a concept it is super lovable: I just hate the branch-office dilemma of either a whole half-rack of underutilized widgets that could support an office twice to ten times the size but just don’t come any smaller except possibly as awful plastic boxes with wall warts and no redundancy; and it’s got that cheerful Microtik pricing and power consumption that isn’t from used enterprise kit 3 generations old.
However, while its concept is the little do-it-all box the RAM and storage are really, really, inflexible for the purpose: even just a plastics kit option that allows you to replace some of the 7mm support trays with ‘every-other’ 15mm support trays would go a long way; and (given that caches and DVRs are two super common use cases) some option to sacrifice SSDs for bulk SATA(either SATA on the ‘every-other’ 15mm or a variant with 3.5in slots) would be an obvious nice to have; and more or expandable RAM would be super helpful: can you get a lot into really thrifty container configs? Sure. If I am looking to equip a relatively small number of branch locations can I afford a fair amount of RAM for what it would cost to either have more talented people configure my containers or have the existing people put extra time into slimming containers down rather than just grabbing defaults and calling it good? Yes indeed; admin time that you’d want doing that can creep up on you fast.
The RAM seems like just a sticking point(though, as noted, the price is right enough just for the switch side of things that you can forgive some sins; it’s just that you are totally blowing past the point if you run out of RAM and need to drag a separate server into it; since the cost of going from zero extra server to 1 extra server will be much higher than going from 32 to 64 or 128GB of RAM; and adding extra RAM will probably be cheaper on the separate server, at least up to 128, possibly not if you need high density DIMMs); but some of the storage-related complaints seem like they could be addressed depending on what, if anything, Mikrotik or others do with those external PCIe lanes.
Normally external PCIe cardcages seem to be obnoxiously expensive(still often worth it if you can now bring that one PCIe card and just a laptop rather than an entire desktop when travelling; or add a GPU or high speed ethernet to a laptop or the like); but expensive enough that they really ruin the numbers on any “let’s use this mini-PC that has thunderbolt or oculink rather than an a mini-ITX or a short depth server that has a couple of card slots!” project; but if Microtik or someone else pop out a few cheerfully priced and relatively simple options that you can use to add some 3.5in SATA mass storage or just a few arbitrary PCIe cards or similar using those external expansion ports that would add a lot of versatility.
Why don’t they just hire @Patrick to make their hardware. If they want to make a NAS or a server it seems dumb not to just hire him and say “tell us what to do” or at least “hey what do you think of this idea?” I’d dream of Patrick designed boxen.
Well, if they hire @Patrick to build the hardware, who’s actually competent enough to fix the software that’s obviously nowhere near what Synology and QNAP offer?
I was thinking they could add some hybrid cages that could either take four 7mm or two 15mm drives. Have two of the U.2 connectors sit slightly further out for the 15mm option and design some slids that can take either one wide or two narrow caddies.
On a different topic, they could have also used SATA drives. 20 of them would give 6*20=120Gbps, which would more or less saturate the 128Gbps of PCIe bandwidth available. Sure, older tech, but would likely use less power and be close in performance.
My biggest issue that’s stopping me from buying one RIGHT now is that it’s crippled by the slow PCIe speeds and bottlenecks. What’s the point of 100Gb/s connections if you can never approach those speeds if the thing is doing pretty much anything else? Hot swap not being a thing is also a no go as I wanted to toss it into my AI environment as a cache but knowing that one drive failure, no matter what, will bring my entire cluster down misses the entire point.
I can deal with the crappy software as I’m perfectly comfortable in a CLI but it also shows that it’s not a fully developed product I think.
I really want to like this thing but for 2k more I can build something similar with none of those issues