Micro or Macro Servers?

Kevin Fogarty pooh-poohs the recent moves to fill racks with hundreds of tiny servers instead of virtual machines running in ever larger brute-force servers.

Indeed, these are two ways of doing the same things, adding cores/processes to the rack getting more throughput/$ or /watt. Unfortunately, Fogarty does not understand scaling.

Scaling is a topic I studied in physics 40 years ago. Consider a cube. Its surface area is proportional to the square of the length of an edge. Its volume/mass is proportional to the cube of the length of an edge. Whether you add cores to a CPU or make a CPU smaller you get more computing power in a given area of chip. By making chips smaller, however, you get way more throughput because each chip can transmit at the same time. More cores on each chip does not scale as well because there is a bottleneck in bandwidth to/from the chip. It’s like having one or four gigabit/s NICs on a server. They are the limiting factor in throughput. With the micro servers, you can have far more throughput in a single box than with the huge multi-core chips. No one is likely to have 128 cores in a chip any time soon but we can easily put 1024 cores in a rack full of microservers and they will have far more throughput.

When I design a terminal server, I always look at the bottlenecks. If your motherboard can move 20 gB/s and your CPU can move 8 gB/s the number of CPUs is the bottleneck, not the power of a CPU. gigabit/s NICs are pretty standard these days. If you want a powerful server, we should be looking at microservers and 10gb/s NICs soon. A powerful CPU is fun/useful but if it is spinning its wheels in a box limited in throughput, that power is a total waste. Have I mentioned throughput/watt? It’s no contest. For example, my Beast uses a 95watt CPU but it is limited to a few gigabits/s throughput to the network. Each microserver might run on 10watts and have gigabits/s throughput. The microserver has 3 times as much throughput per watt.

For a database, througput to storage may be more important than throughput to the network, in which case macroservers will continue to make sense, but there are many more file servers than data servers. Microservers make a lot of sense and ARM will be king of them because the chips are so much smaller than x86.

Fogarty has a point that expanding virtual servers is easier through Moore’s Law but Moore’s Law works for microservers too. One can double the number of microservers in a rack through one step or Moore’s Law. In the case of a macroserver, one could change a CPU in a socket whereas for microservers, one might need to change a small motherboard. Eventually, those tasks will be nearly identical as the microservers and their local storage will be made smaller.

About Robert Pogson

I am a retired teacher in Canada. I taught in the subject areas where I have worked for almost forty years: maths, physics, chemistry and computers. I love hunting, fishing, picking berries and mushrooms, too.
This entry was posted in technology. Bookmark the permalink.

2 Responses to Micro or Macro Servers?

  1. The microservers tend to have some RAM, not a lot, but some by each CPU. They just have the response time of their RAM to worry about. The macro servers may have far more RAM and faster RAM but that access time is far less than a round-trip on the web. Some of the early ARM systems had really slow RAM but these days it seems to be within a step or two of “normal” PCs where one has pretty fast RAM in order to try to please many processes. The microserver architecture may not be the fastest to respond but in terms of cost per process or processes per watt they are in the game.

  2. Alex says:

    Robert, you speak about throughput, but latency could also be an issue in largely multicore systems. To consider is access to RAM, which is slow, compared to the CPU speed, and having multiple mobos with dedicated RAM for each could be better than a lot of cores all competing for the same RAM… at some point, the refresh time of the DRAM + row/col select time will kick in and stop things from improving linearly.

Leave a Reply