Evolution Of My Plan To ARM IT

The on-off-on status of the LeMaker Cello makes my original plan to replace my current energy-guzzler with that motherboard difficult to say the least. There isn’t a single ARMed motherboard on the market that matches my needs. In addition, I’ve done some maths. The energy consumption of Beast’s CPU over the last decade has almost certainly cost more than the hardware… Remember, some of that usage was off-grid, sucking in $1/kWh diesel-electric power. Oh, the pain…

So, my thinking has changed. There are a number of smaller/simpler boards that do just fine as clients like Odroid-C2 ($46 USD) and FireFly-RK3399 ($219 USD with 4gB RAM). They aren’t a problem. It’s the server end of things. I think I have a solution. The 96Boards people have a “set-top” offering, Poplar ($120 CDN, delivered), with a PCI-e slot that can hold some SATA device connections. It has gigabit/s Ethernet but also USB 3 connectors. I can use the Ethernet to do NFS, etc. to the clients and the USB 3 can be bridged to multiple boards to act as server. Unfortunately these boards are 28nm so the savings in energy is not great. The FireFly-RK3399 could be a decent server if the USB 3 bridge is solid. I’ve been testing a PCI-e SATA card and it’s solid. Total power consumption would be half what Beast uses but this could be a good placeholder until some better ARMed systems happen.

About Robert Pogson

I am a retired teacher in Canada. I taught in the subject areas where I have worked for almost forty years: maths, physics, chemistry and computers. I love hunting, fishing, picking berries and mushrooms, too.

This entry was posted in technology and tagged , , , , , , . Bookmark the permalink.

37 Responses to Evolution Of My Plan To ARM IT

  1. oiaohm says:

    Deaf Spy
    Because, Robert, USB 3 is specifications, with theoretical parameters. Actual results depend greatly on the actual implementation.
    I gave forum posts where people are testing it in real world. USB 3.0/USB 3.1 rev1 with the same make of board being odroid XU4 and massive variation.
    https://forum.odroid.com/viewtopic.php?f=99&t=15196
    So its not that most of these arm boards cannot get decent USB 3.0 performance. The quality if the usb 3.0 to sata bridge is important the quality of the USB hub is import and quality of the drive hardware used is important.

    Yes the forum tests of the odroid XU4 tests shows the WD my book drives as one of your worst possible choices. So nice uniform hardware test.

    Was I looking at theoretical numbers alone no Deaf Spy. Decent arm hardware is able to achieve a lot.

    http://www.hardkernel.com/main/products/prdt_info.php Please note odroid XU4 is not a really modern arm cpu design.

    Firefly-RK3399 and the poplar cpu serousally leave odroid XU4 in the dust.

    http://www.anandtech.com/show/9184/arm-reveals-cortex-a72-architecture-details
    The A72 in the Firefly are 3 times faster than the A15 in the odroid. XU4. The 4 Cortex-A53 are in fact able to process more than the 8 A7 + 2 A15s in the odroid.

    Deaf Spy when the older arm hardware could achieve could achieve quite respectable USB 3.0/3.1rev1 performance and the newer hardware is in fact faster and benchmarked showing that. All it says is you are a idiot DeafSpy who did not know the topic. I do know there are Intel atoms with USB 3.0 ports that cannot keep up with a odroid XU4 board let alone anything current arm.

    odroid XU4 is from 2015. Basically Deaf spy you are getting to be 2 years out of date.

  2. Deaf Spy wrote, “USB 3 is specifications, with theoretical parameters. Actual results depend greatly on the actual implementation.
     
    Just like any IO bus out there since times immemorable.”

    USB 3 is also achievable with the usual components.

  3. Deaf Spy says:

    Why would anyone be surprised by USB 3 being USB 3?

    Because, Robert, USB 3 is specifications, with theoretical parameters. Actual results depend greatly on the actual implementation.

    Just like any IO bus out there since times immemorable.

  4. Deaf Spy wrote, “get a good USB 3.0 hub and give it a spin! Then surprise us with the results.”

    Why would anyone be surprised by USB 3 being USB 3? Besides, it’s not a hub I will be using but a bridge, something quite different. You can’t connect two USB hosts together without one.

  5. oiaohm says:

    Deaf Spy

    Still, how will this help you when the USB 3 on your ARM home-tv board turns out inefficient to handle a single drive, let a lone a bunch of drives?
    https://www.raspberrypi.org/forums/viewtopic.php?f=46&t=32454
    Really USB2 on arm hardware gets 20-30 MB/s

    I have a small cheap computer, with Atom Z8300 quad-core CPU at home, and an external 4TB WD Book hooked to it. It is based on a decent Intel chipset, and still can’t manage more than 25MB/s.

    Really DeafSpy you are basing you numbers of broken intel hardware.

    https://forum.odroid.com/viewtopic.php?f=99&t=15196 older slower arm hardware than what robert is looking at is 3 to 4 times faster than USB2 using a USB 3 port.

    You are seeing people getting 120 MB/s+ on a USB 3 connected drive to arm hardware.

    USB 3.0 works as long as it not the early ones made by Intel and as long as you usb-Sata bridge and usb 3.1 rv2 hub are quality.

    Deaf Spy reality if you had checked out the benchmarks on the arm boards with USB3 you would have noticed they beet out of old intel atom USB 3 performance by such a margin its not funny as long as quality key parts are used.

    http://rockchip.wikidot.com/rk3399

    RK3399 cpu in firefly is USB 3.0 rv1 5GB/s max. So it will run a single 5400 drive flat stack no problems and run 7200 quite well. A good hub adds very little overhead.

    The reality is 5400 drives really don’t need more performance than Sata II can provide.

    Poplar is annoying because Hisilicon Hi3798CV200 soc has Sata but the Poplar board does not put a port on it. There are other Hisilicon Hi3798CV200 better configured.

    Deaf Spy and both of those socs if you look around have performed benchmarks using their USB3 ports for drives. Basically stop being a moron and making stuff up.

  6. Deaf Spy says:

    Nope, the second value is reading from the surface of a 1gB 7200 rpm drive.

    Gee, haven’t noticed that. 🙂

    Still, how will this help you when the USB 3 on your ARM home-tv board turns out inefficient to handle a single drive, let a lone a bunch of drives?

    But, Robert, why are you arguing? Just buy one such board, get a good USB 3.0 hub and give it a spin! Then surprise us with the results.

  7. oiaohm says:

    Deaf Spy there are benchmarks going through proper usb3.0 and 3.1 hubs.
    https://www.everythingusb.com/speed.html

    https://kb.sandisk.com/app/answers/detail/a_id/8142/~/difference-between-sata-i%2C-sata-ii-and-sata-iii

    USB 3.0 is 5Gb/s or 4.8Gb/s depending on what manual you read with a real world of 3.8Gb/s . What makes it faster than Sata II and fairly close to Sata III

    So USB 3.0 connected drive you should be able to get at least 300MB/s if the drive can keep up and the controllers are good.

    Timing buffered disk reads: 1546 MB in 3.00 seconds = 514.98 MB/sec
    You can get this on a USB 3.1 rev 2 Grece. USB 3.1 rev2 is faster than Sata III before 3.2.

    To be faster than USB 3.1 rev 2 you need to be SAS-3+ or Sata 3.2+.

    Now there is differences in quality with USB hubs you would be after a USB 3.1 rev 2 hub for drives. Also you need decent usb to sata conversion boards.

    If the USB 3.0 port truly support 3.1 revision 2 is in fact faster than Sata III.

    http://hdd.userbenchmark.com/Compare/WD-Blue-4TB-2015-vs-WD-Red-Pro-6TB-2015/3523vsm139230

    The reality here is nasty a WD Blue on a decent usb 3.0 to sata converter is not going to push the USB 3.0 port at all. Performance should be almost identical with it sitting on a USB 3.0 port even with 2 of them on a decent USB 3.1 rev2 hub as to the two WD blue directly connected to to sata on the motherboard.

    WD Reds even the new ones are not fast enough to keep on filling USB 3.1 rev2 can push USB 3.1 rev 1 to limit.

    This is the problem sometimes is not worth the effort to go sata when you have USB 3.1 rev2 with decent usb to sata board or you are on usb 3.1 rev 1 and the drive is something like a WD blue that suxs in performance with a decent usb to sata board. Why because connecting to direct sata is not giving any more transfer speed.

    There have been issues with some of the intel atoms with early usb 3.0 controllers of using USB 2.0 chit chat when it was not required and results in doing stupid connection speed changes. Yes how to see 25 Mb/s when you should be seeing 80+ its either the chip you are using has bad USB 3(ie Intel) or the usb to sata converter at the drive is trash.

    Remember intel was one of the first implementors of USB 3.x their early stuff had some nice nasty performance killing design faults.

  8. Grece says:

    Nice values Robert, but I was referring to speeds over NFS. Not local read speeds on your RAID0 array; please pay attention.

    Since you wanted to gloat about local read speeds, here goes.

    sudo hdparm -tT /dev/sda

    /dev/sda:
    Timing cached reads: 23568 MB in 2.00 seconds = 11795.39 MB/sec
    Timing buffered disk reads: 1546 MB in 3.00 seconds = 514.98 MB/sec

  9. Deaf Spy has a problem reading, too:“Benchmarking cache performance of hard drives?”

    Nope, the second value is reading from the surface of a 1gB 7200 rpm drive.

  10. Deaf Spy says:

    Just saying.

    Benchmarking cache performance of hard drives? Totally meaningless.

    Even the pathetic Banana Pi Pro will read

    My point, Robert, was to make you think that USB 3.0 is by far not any solution to your case.

    But, if you are so confident, why don’t you do it, ok? Buy this board, plug your drives via a USB hub, and come back to report. We’re all eager to hear your experience.

    Or, are you just idle-babbling?

  11. oiaohm says:

    wizard emiritus
    You mean like these?

    https://www.ebay.com/i/192269523813?chn=ps&dispItem=1

    Not quite. The good ones are PCI-e x16 host slot then a PCI-e x16 full PCI-e x8 slot then a PCIe x4 slot.

    These can be almost pure electrical logic.
    If you have PCI-e 16 in alone it gets PCI-e x16. If you have a card in the PCI-e x8 and in the PCI-e x16 both get PCI-e x8 as long as the PCI-e x4 is free. If something is in the PCI-e x4 the PCI-e x8 slot comes x4 and the PCI-e 16 is still PCI-e x8.

    These cards are normally made matched to motherboard/SOC. So getting your mits on them is a little tricky particularly when you don’t want 10000 of them and they make that single port a lot more useful.

  12. oiaohm says:

    Grece the WD my books the premade caddies. They have 5400 blue drives in them.

    https://www.jbhifi.com.au/computers-tablets/all-computers–tablets/wd/wd-my-book-4tb-external-hard-drive/332946/

    https://www.wdc.com/products/internal-storage/wd-blue-pc-desktop-hard-drive.html#WD40EZRZ
    I will give you typo kind of. The drive inside caddies is a 5400 rpm on outside. Broken open lot of them are really only spinning up to a max of 5200. So 200 rpm under spec.

    There is no 5200 rpm drive dumb ass.
    There is no disk with a label of 5200 rpm is correct. There are discs that don’t go up to their rated RPM. Interesting enough you normally only find them inside the pre-assembled stuff.

    WD 4TB drives are rock-solid. << please note I did not say the drives lose data just they don't perform as well as they should.

    You buy a WD Blue PC Desktop Hard Drive 4G 5400 harddrive you get a so called WD Blue PC Desktop Hard Drive 4G 5400 out of a WD my book and notice they don't perform the same. The one out the caddy is slower. Some are worse that the SATA to usb3 are part of the controller board.

    I get 120MB/s with my WD 4TB reds, I have thirteen of them or so in my one NAS.
    Grece WD don’t put reds in my book caddies. There is 20 dollars difference between a WD 4TB red(that is 7200) and WD 4TB blue that is 5400 if to spec. But there is major differences in performance.

    This is a common trap those using WD Reds recommend WD Caddies without being aware they are a lower grade product.

  13. Grece wrote, “I get 120MB/s with my WD 4TB reds”.

    Hmmm…
    hdparm -tT /dev/sdd
     
    /dev/sdd:
    Timing cached reads: 3046 MB in 2.00 seconds = 1522.45 MB/sec
    Timing buffered disk reads: 624 MB in 3.00 seconds = 207.78 MB/sec
    ls /dev/disk/by-id

    ata-ST1000DM003-…

    Just saying.

  14. Grece says:

    4TB WD Book hooked to it.<< Piece of crap. Its a 5200 hard drive is inside. I have done data recovery on them those drives ripped out and directly connected to sata perform badly as well.

    LOL…. the WD 4TB drives are rock-solid. There is no 5200 rpm drive dumb ass. Data recovery? On who’s disk, yours?? Do tell the story on that!

    I get 120MB/s with my WD 4TB reds, I have thirteen of them or so in my one NAS.

  15. The Wiz wrote about pcie expansion.

    That’s interesting but I’d like more lanes. My beloved card can run 4 sata and uses two lanes.

    See http://mrpogson.com/2016/06/30/adventures-in-sata/

  16. wizard emiritus says:

    “Hisilicon Hi3798CV200 is interesting it was made for htpc/media server with the presume you know how to get hands on a PCI-e x16 multi port riser that is quite hard unless you are a OEM builder.”

    You mean like these?

    https://www.ebay.com/i/192269523813?chn=ps&dispItem=1

  17. oiaohm says:

    https://www.amazon.com/IO-Crest-Controller-Components-SI-MPE40125/dp/B072BD8Z3Y
    Dylan
    Also, if any case, the PCI-E slot is just there to plug a TV tuner card in.
    No the FireFly-RK3399 includes 2 PCI-e mini slots. One is positioned on the top side of board for solid state drive or like the above sata card.

    There are quite a few raid cards that are compatible. Hisilicon Hi3798CV200 is interesting it was made for htpc/media server with the presume you know how to get hands on a PCI-e x16 multi port riser that is quite hard unless you are a OEM builder.

    http://www.tomshardware.com/forum/297224-32-speed-external-usb3-internal-sata

    Deaf Spy USB 3 is not that slow. In fact on some of the boards that have PCI-e full or mini and USB 3 its can be not worth cost to go after the PCI-e if you are using spinning rust and decent USB3 to Sata bridges. If you using a solid state hard drive then there will be a major difference between using a sata port or using USB3. The fact USB 3 has a on board control makes it way different to USB2. USB3 using a hub is safer than SATA using a multiplier for about the same performance with spinning rust.

    4TB WD Book hooked to it.<< Piece of crap. Its a 5200 hard drive is inside. I have done data recovery on them those drives ripped out and directly connected to sata perform badly as well.

    WD usb to sata bridge performance is crap the drives they put in there externals are crap. You will notice Cioby Jun 26, 2015 back in that link was complaining that the speeds the person was seeing was impossible with WD premade hardware. Buy a caddy with a decent sata bridge and a WD drive and it cheaper in a lot of case and performs better than the WD premade stuff.

    Atom Z8300 you have to watch carefully I have seen 25 mb/s on their sata 3 port with a sata break out you notice its talking USB 2 not USB 3. You should be seeing around the 80Mb/s on USB 3 on a Atom Z8300 if everything is working right. Now if the Atom has failed backwards to USB 2 getting decent performance out the USB 3 port is now impossible.

  18. Kurkosdr wrote, “your plan to use the tiny CPU of an Android TV board to handle everything from USB 3 to NAS to DNS to HTTP server has been sufficiently ridiculed”.

    That’s not the plan at all. Garbage in and garbage out of your argument. I propose running storage on the set-top box linking to a more powerful board for the applications. It will be a NAS, in effect.

  19. Kurkosdr says:

    Kurkosdr changed the subject rapidly:

    I think that your plan to use the tiny CPU of an Android TV board to handle everything from USB 3 to NAS to DNS to HTTP server has been sufficiently ridiculed. If you think this isn’t true, we can ridicule it a bit more.

  20. Kurkosdr changed the subject rapidly: “Another quality LeMaker product with excellent OS support.”

  21. Kurkosdr says:

    Even the pathetic Banana Pi Pro will read 46MB/s from a 160gB 7200rpm drive AND transmit it via Ethernet…

    I looked up the specs. OS support: Android 4.2 / 4.4

    Another quality LeMaker product with excellent OS support.

  22. Deaf Spy prattled, “On such a board, you’d be happy if you get 25MB/s from a single your HDD you plug in.”

    Even the pathetic Banana Pi Pro will read 46MB/s from a 160gB 7200rpm drive AND transmit it via Ethernet… See https://www.htpcguides.com/raspberry-pi-vs-pi-2-vs-banana-pi-pro-benchmarks/

    You forget I have an Odroid-c2 and it meets specs on connectivity with CPU power to spare. The chips I’m considering have 4 or more cores and higher clocks. One can easily run a hard drive hard while another does something else.

  23. Deaf Spy says:

    few SATA connections … Connecting them by USB 3 makes sense

    That’s exactly one of your problems, Robert. You can’t do that and get the desired effect. Forget about the theoretical throughput of USB 3.0 . On such a board, you’d be happy if you get 25MB/s from a single your HDD you plug in. Plug in a few via a hub, and things will deteriorate very quickly.

    How do I know? I have a small cheap computer, with Atom Z8300 quad-core CPU at home, and an external 4TB WD Book hooked to it. It is based on a decent Intel chipset, and still can’t manage more than 25MB/s.

    But hey, why don’t you buy one of these boards and prove me wrong?

  24. Kurkosdr says:

    FYI – This is an example of a real low cost server

    http://www.dell.com/en-us/work/shop/productdetailstxn/poweredge-r230

    Don’t frighten Pog with that price tag. A basic PC would still be much better than the Android-TV-board-with-USB3 that Pog wants to use as server.

  25. wizard emiritus says:

    ” If I want a small server in my home, why can’t I use anything with the relevant resources to get it? ”

    You can do and believe whatever you want, but it still doesn’t actually make what you have chosen for yourself a server in the sense that that most everyone else does.

    FYI – This is an example of a real low cost server

    http://www.dell.com/en-us/work/shop/productdetailstxn/poweredge-r230

  26. Deaf Spy parroted, “this is not a server board, per se”.

    How is that relevant? If I want a small server in my home, why can’t I use anything with the relevant resources to get it? I need Ethernet for NFS and HTTP and DHCP and DNS. I need a few SATA connections which I can’t get with one of the newer embedded ARMed boards but I can get with another. Connecting them by USB 3 makes sense. Consider it a NAS for the server.

  27. Deaf Spy says:

    Which is weird, since this is not a server board, per se.

    Of course it is not. But good luck explaining this to Old Robert.

  28. Dylan says:

    Also, if any case, the PCI-E slot is just there to plug a TV tuner card in.

  29. Dylan says:

    Hey, Robert, does this server thingie have enough caches for your needs? What is its power consumption?

    Which is weird, since this is not a server board, per se. When’s the last time you ever see a server with HDMI 2.0a output with HDCP 2.2, S/PDIF optical audio, onboard H.265 HEVC 4K UHD video decoding at 60FPS, WiFi(for when you don’t have access to wired ethernet next to your TV) and a integrated IR receiver(for controlling the HTPC, err server, from your couch)?

  30. Grece says:

    Robert will dick-around looking for this and that, never getting around to replacing everything. The his board crashes, taking along is RAID0 array as well.

    Just get a Supermicro server board running an E3 Xeon and quit playing around.

    7W — power off; idle consumation, the IPMI is alive
    39W — power on; Linux OS is idle, IPMI sensor readings: cooling FAN works with 1755 RPM — relatively quiet; CPU temperature is Low
    45W to 60W — power on; moderate Linux OS usage, load average: 1.53; installing 200 new packets via “apt-get” IPMI sensor readings: cooling FAN works with 1755 RPM — relatively quiet; CPU temperature is Low
    130W — power on; full stress by “stress –cpu 16 –io 8 –vm 8 –vm-bytes 1780M –hdd 4″
    load average: 36.00; I/O load: 100%, mostly write; CPUs busy @ 100%, 70% user, 30% system, all CPU cores are utilized RAM: about 95% used, 30% cached; network load: 22 Mbit/s constant SSH transfer IPMI sensor readings: cooling FAN works with 3100 RPM — much noisy; CPU temperature is Medium

  31. Kurkosdr wrote, “he will have to make do with USB 3.0, which as we all know is a CPU hog with high theoretical speeds but low real-life speeds in low-end hardware”.

    USB 3 is a standard means of interconnection. It could be done by hardware but usually there is some software/firmware controlling that process. So, it absorbs some CPU but all over the world we have CPUs idling so it’s no big deal. Further, how many of the 4 or 6 cores do you think it takes to run USB 3 with >gHz clocks and 64 bit RAM? Further, what are the other cores going to be doing other than moving data to/from devices? Do you think they will even warm up?

  32. oiaohm says:

    Kurkosdr need to check the facts again. proper PCI-E lanes Cello has proper PCI-E lanes .

    http://b2b.gigabyte.com/Server-Motherboard/MP30-AR0-rev-11#ov
    So does this older Arm board have proper PCI-E lanes.
    http://www.96boards.org/product/poplar/
    Yes this has proper PCI-E lanes. Issue has been PCI-e card firmware.

    This would get him the best possible energy efficiency you can get with proper PCI-E lanes.
    LOL no. The reason why more and more arm chips are used in items running shop displays is that x86 is not the best energy efficiency in all cases any more.

    With the emulation in the hi-silicon stuff the PCI-e stuff is working.

    And this means he will have to make do with USB 3.0, which as we all know is a CPU hog with high theoretical speeds but low real-life speeds in low-end hardware, and he apparently the whole setup is a substitute for a proper server. Or thinks it is guaranteed to work.
    Robert put the link to something with proper PCI-e. I gave the link that informed you that the hisilicon hardware works. So this is you just saying garbage.

  33. Kurkosdr says:

    Now, in regards to Pog’s problems, they can be solved by any basic recent x86 board on the market and a recent x86 CPU. This would get him the best possible energy efficiency you can get with proper PCI-E lanes. But you see, Pog has principles™, he can’t just buy a x86 board because x86 is a patented architecture, he wants to buy an ARM board because ARM is a patented architecture (patented by the GoodGuys™ however, however they are defined in the movie playing in Pog’s head).

    And this means he will have to make do with USB 3.0, which as we all know is a CPU hog with high theoretical speeds but low real-life speeds in low-end hardware, and he apparently the whole setup is a substitute for a proper server. Or thinks it is guaranteed to work.

    Now excuse me, I have to find a house to rent with insulation made out of Llama’s wool and melted plastic, because DOW Chemical are a bunch of BadGuys™ and I couldn’t think of financially supporting them by living in a house with their insulation boards.

    There isn’t a single Llama-ed house on the market that matches my needs.

  34. Kurkosdr says:

    The cello boards that port the big problem AMD run into that they could not get that work.

    More “formal English” from Ohioham.

  35. Deaf Spy says:

    Hey, Robert, does this server thingie have enough caches for your needs? What is its power consumption?

  36. Deaf Spy says:

    Come on, Robert, do it!

  37. oiaohm says:

    http://www.workofard.com/2017/03/project-dogfood-my-arm64-desktop/

    Interesting point Hisilicon worked on qemu support to support pci-e on arm completely.

    The cello boards that port the big problem AMD run into that they could not get that work. Remember Hisilicon boards have on board video cards the Cello was lacking that. So the Cello had a fatal flaw when the PCI-e bus would not function with PCI-e graphics cards.

    Basically it sounded so simple PCI-e slot on an arm. Gigabyte Arm board had dual PCI-e x16 right this will be simple. One major catch Gigabyte board had on board vga port so graphics cards did not need to work in the PCI-e bus.

    AMD made a mistake with Cello board and its only about fixed now. Thinking Hisilicon is behind the fix it going to be interesting going forwards.

    Interesting point without the Cello board we could still be in the location where PCI-e graphics cards on arm64 were impossible option.

    Of course I am waiting to see boards with proper changeable ram and graphics cards. 4g of ram for 6 cores does get kind of questionable.

Leave a Reply

Your email address will not be published. Required fields are marked *