Adventures In SATA

Nothing much to report but I received my new SATA controller card in the post box yesterday. TLW was languishing in bed so I had an opportunity to reboot Beast III in order to install the card. The PCI-e slot I thought I would use was the wrong one. Not enough lanes. The big one worked fine. Beast booted up as if nothing had changed but now
lspci|grep Marv
02:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller (rev 11)

No need to use that driver CD included in the box… ๐Ÿ˜‰
The card is an IOCrest SI-PEX40062 with Marvell 88SE9235 chip, two lanes of PCI-e V2.0, and SATA 3. They claim it’s hot-swapable so I unmounted one of my new 1TB hard drives, unplugged its SATA cable, connected one of the two new SATA cables that came in the box and plugged the other end into the card. Zero problems. I remounted the drive and was good to go. hdparm showed a similar speed as the SATA connection on the motherboard. For $49.71CDN delivered to my mailbox, it’s all good. The new ARMed server board will love this.

A postal strike/labour dispute is expected this weekend so if there’s nothing in the mailbox today, I may be out of luck for a while. Some parts for my new IT-system will come by courier.

About Robert Pogson

I am a retired teacher in Canada. I taught in the subject areas where I have worked for almost forty years: maths, physics, chemistry and computers. I love hunting, fishing, picking berries and mushrooms, too.
This entry was posted in technology and tagged , , , , . Bookmark the permalink.

15 Responses to Adventures In SATA

  1. oiaohm says:

    (with raid5 you MUST replace with the same model, AFAIK).
    dougman that is in fact not true. Standard conforming controllers you can in fact change models and brands to another standard conforming or drop back to software even in Raid 5. Basically lot more durable and simpler to replace if you have to. This is normal like “LSI or Areca”. One basic rule do not use a older card than the one you are replacing as it might be older edition of standard. Newer version of standard is backwards compatible older version of standard is not forwards compatible of course..

    Intel is not to standard but they have documented their raid 5 and all the matrix cards and software emulation of it does it the same way. So changeable between models if required because no matter the intel matrix model they all do raid 5 the same way. Only one basic rule do not use a older card than the one you are replacing same reason due to revisions.

    Adaptec don’t use raid 5 to standard but all their modern controllers do it the same way so are changeable with other modern Adeptec and in disaster Adeptec can provide a software rebuild option so standard sata/sas not raid can be used to recover the data.

    Then you get into the horrible brands with complete custom stacks and no good support tools. Some of these swapping with the same model is still complete failure.

    True I am likely to stay away from raid 5 to have more controllers running to standard so making drives way more swap between controllers. Adeptec are to standard in every mode bar raid 5/6 with raid 6 choice between standard and not standard.

    Mind you Adeptec raid 6 is super odd.
    http://ask.adaptec.com/app/answers/detail/a_id/15313/~/what-are-the-differences-between-the-adaptec-raid-6-and-raid-6-reed-solomon%3F

    Yes the standard raid 6 is raid 6 reed solomon. As adaptec notes themselves is adaptec own will be picked up and auto converted by a raid 6 reed solomon supporting raid controller in a rebuild if it intact.

    More and more Raid controllers are appearing that are pure to standard so none of this same model or else problem if you buy a standard conforming raid controller. Like adaptec is slowly ending the non standard xor raid 6 and their raid 5 will most likely go the same way. Raid 6 is designed to deal with dual disc failure.

    Heck, for that matter, you can take every drive out and plug them into a completely different motherboard with different controllers for each drive and get it to work (provided your new hardware works with unraid).
    Since Linux kernel implements all the standard conforming RAID in software mode you don’t need a raid controller to read discs from standard conforming implementations(hardware or software) and can use completely different controllers for each drive just this means using software raid. Yes standard raid controllers must have a option to switch back to raidless sata/sas to allow software recover.

    This is the things that have changed. To standard raid on disc equals choice of brand and range of recovery options.

    Basically it started with
    DDF (Disk Data Format) defined by SNIA
    Then SNIA has gone on and defined how each RAID mode on disc is meant to exist as well. So SNIA conforming raid controllers are very compatible with each other no matter if they are software or hardware or what vendor they are. If buying a raid controller I see limited reasons to consider anything not conforming.

  2. dougman wrote, “Raid5 means 2 dead drives = ALL data lost.”

    That can be true in a poorly designed system. In a system with regular backups even the rarest hardware failures only cost one period’s worth of data. There are places where that is not acceptable but many more, like browsing the web using web-applications a lot where the clients don’t need anything like that kind of reliability. Of course, there’s nothing wrong with being totally paranoid but it costs extra time/money/space and it’s often a total waste. In ten years, Beast has lost exactly one hard drive. I was using RAID 1 for years to avoid problems and to serve better multiple users but now that’s not necessary. I can make more efficient use of my hardware by using RAID 0 or even single drives. They are sufficiently reliable. IMHO a good hard drive is much more reliable than TOOS. I often found PCs that would not boot in schools but rarely encountered a dead hard drive. I think I found two in all my years of teaching, one was infant mortality and another was some accident of old age. Many large users of hard drives report failure rates of less than 1% per annum. One hard drive could last my lifetime. A bit of backup or redundancy could last several lifetimes. TLW has not lost any file since she started using GNU/Linux. She used to lose several every year when TOOS crashed. Reliability of hard drives is not the smallest worry but there are many more for folks who use TOOS. Mostly GNU/Linux works as intended and stays out of people’s way.

  3. dougman says:

    After reading this tripe, I cannot even begin to form a response.

    But suffice to say, I do not even USE raid anymore I prefer the unraid system, inasmuch as you are not as limited. You can add/remove/change drives at will, and you are not in a major “uh-oh” situation if your HD/Raid controller ever fails (with raid5 you MUST replace with the same model, AFAIK). With unraid, you can take out any drive (except parity) and plug it into any computer that can read xfs and get the data off. Heck, for that matter, you can take every drive out and plug them into a completely different motherboard with different controllers for each drive and get it to work (provided your new hardware works with unraid).

    One parity drive WILL protect all the other drives from ONE HD failure. This is the same as raid5 (failure wise) with the added bonus that if TWO failures occur, you only lose data on the two drives that failed (unless one is parity, then you only lose one drive) and every other disk is still usable. Raid5 means 2 dead drives = ALL data lost.

  4. oiaohm says:

    I STILL can remove the drive and pull data off it as it formatted in XFS.
    Hardware raid odd mapping or encrypted. You remove the drive connect to another system it does not read able dougman.

    What the drive is formatted is totally not related to this problem. As file system formats is a level above ddf or intels or custom crap metadata for raid and that is a level above encryption.
    Layers of layouts
    On a active raid controller
    1 File system.
    2 Raid storage metadata.
    3 Encryption
    4 drive controller sectors.

    On a sata controller
    1 file system
    2 drive controller sectors.
    http://www.snia.org/tech_activities/standards/curr_standards/ddf

    If you cannot read the Raid storage metadata you may not be able to get the sectors stored in the drive back into the correct order to make any sense to the file system driver. Hardware/Software Raid controller storage meta data in software is like looking at.
    1 filesystem
    2 LVM
    3 dm-crypt
    4 drive
    Yes exactly the same problem. Of course the encryption bit can be dropped. But if you cannot read the meta data layer of LVM/DDF/Intel Matrix knowing what the file system is above it is next to worthless.

    This is why with a hardware raid controller active even a single drive connected can ruin your day if it a incompatible type. Yes a raid 0 with one drive still ends up with hardware raid controller meta data written messing up reading back that drive on a normal sata controller because there was extra data written.

    If you want the means to be sure to disconnect a drive and connect it to whatever a plain sata or sas controller is what you are looking for.

    Basically there are many ways a hardware raid controller can kick you in your teeth.

  5. dougman says:

    blah blah blah…

    Hardware, software, raid controller failures.. eh….

    I STILL can remove the drive and pull data off it as it formatted in XFS.

  6. oiaohm says:

    No need for raid recovery. Just remove the drive, and plug it into another computer and DD the information off to a new drive. But thatโ€™s the point of a parity drive to begin with.
    dougman something to be aware of some hardware raid controllers format the drive using methods that they can only read. So hardware raid controller go splat reading your data back without identical version or software option mode your data is locked up badly at times.

    Software raid for recovery mode is that the hardware raid controller is dead and you still need to read the data.

    http://www.intelraid.com/files/SED_brief.pdf
    Also that idea of DD to another drive as solution to problem does not work in Secured Raid setups at times. Because drive serial number no longer matches so hardware raid controller can refuse to decode drive you have copied the data to because it serial numbers wrong. Even in a Raid 1 can have both drives develop a fault a exactly the same time. Yes this is a case when you are thankful for a software mode where you can do stuff that the hardware raid controller refuses to-do.

    As I typed it directed depends on what the Hardware Raid controller is. Now Hardware raid controller encrypting that you cannot in recovery mount the drives with software controller can be instance of data loss if the controller stops.

    One drive could fail, and I just swap drives and rebuild parity and continue on.
    Wake up the issue I am referring to is not the a drive failure but the controller failure. So all drives can be fine but the hardware raid controller has dropped over dead.

    If using hardware raid controller means you cannot recover you data with the drives without the controller you are safer to use software raid.

    The first format is the DDF (Disk Data Format) defined by SNIA as the “Industry Standard” RAID metadata format. When a DDF array is constructed, a container is created in which normal RAID arrarys can be created within the container.

    The second format is the Intel(r) Matrix Storage Manager metadata format. This also creates a container that is managed similar to DDF. And on some platforms (depending on vendor), this format is supported by option-ROM in order to allow booting.
    https://raid.wiki.kernel.org/index.php/RAID_setup
    This is what Linux software raid supports. Now not all hardware raids use either Intel or DDF. So some random brand using some custom format non standard meta data for the raided discs equals mega-trouble if the controller dies.

    This is the reason why I don’t say use hardware raid all the time. Hardware raid controller if you have it must conforms to either DDF or Intel if not don’t use it instead use Linux software raid at least it standard conforming so making recovery simpler if something goes wrong.

    Of course a hardware raid that is DDF or intel with encryption wrapped over the top may also be a recovery nightmare if you cannot use like Linux normal decryption on the drives with the keys in software mode. So its not just use a hardware raid controller it comes to how standard conforming the controller.

    Sata multipliers can also provide hardware raid options but you have to be insanely careful with those as well. Why because 1 drive stops on some models of Sata multipliers and all drives on the multiplier becomes not accessible. Yes some hardware raid pci-e cards have this response as well. So hardware raid controllers don’t always equal dependable why more complex functionality more locations for a developer/designer to mess it up. Stupid enough some of them using them in single drive mode with the raid function disabled they don’t lock up with a single drive locks up. So even having a hardware raid controller card does not mean you will not be using software raid because due to the hardware raid design that is more dependable to use software raid.

    Use hardware raid if all the questions about it answer the right way. If you cannot confirm something about a hardware raid that it standard the safer path is software raid standard conforming as Linux Kernel offers.

  7. dougman says:

    No need for raid recovery. Just remove the drive, and plug it into another computer and DD the information off to a new drive. But that’s the point of a parity drive to begin with.

    One drive could fail, and I just swap drives and rebuild parity and continue on.

  8. Dr Loser, just to be cruel, wrote, “Can we assume, then, that you will not be investing your hard-earned money in that nasty Cello thing”.

    Not this month or next. Probably about August/September. I can play with all the other components of my new IT-system from Beast III. They promised Q2. Perhaps that was a typo and it’s actually Q3… ๐Ÿ™

    In July I plan to buy a fistful of 1TB SATA drives. I will play around with different file-systems and software RAIDs. I might even try an ARMed virtual machine on the new array before buying the Cello. It would be so cool if I could just move a RAID array to the new board and have it work. What are the odds? RedHat says they’ve tried to get them to work just like legacy PCs. It’s worth a shot and public disc images with Ubuntu GNU/Linux are available as a backup plan.

  9. oiaohm says:

    dougman it does depend on the platform if software raid is a dead end or not.

    Arm64 SOC really mess with the idea of software raid being dead. Amd A1100 for example it supports a max of 14 Sata 3 ports. There is no hardware raid controller on all 14 ports. I can list other Arm64 SOC that are the same.

    Someone prototyping up solutions for most of the Arm64 platforms would have very limited interest in raid controller.

    http://www.diskinternals.com/raid-recovery/how_to.shtml
    There is a direct downside to a hardware raid of some random brand. If you cannot get a hardware raid with a software raid mode recovery its not worth having because that is a key mode when everything goes wrong.

    I would not have the rule of always use a hardware raid controller.
    1) It depends what the hardware raid controller is if you should use it or just use software raid directly based on how recoverable it will be in case of major issue.
    2) Like some Arm64 socs the fastest data transfer is to non raid ports by a large margin(major cause is absolutely direct link to memory controller). So the overhead of the software raid is less than going out to a hardware raid card in those cases. CPU processing time paid to avoid IO waiting.
    3) Power configuration of hardware. How protected is the main board from power outage and how does this compare to what the raid controller offers.
    4) Yes the quality of the software raid is also a factor.

    Case by case base. After being burnt by either hardware or software raids the person has a habit of going I will never use the one that burnt them ever again and not properly balancing up the risks. So one case hardware raid will be correct answer anther case will be software raid correct answer all based on the 4 points.

  10. dougman wrote, “Software RAID is a dead-end.”

    Well, for a tiny % of CPU power you get hardware independence. ie. I moved software RAID array from Beast II to Beast III and it worked on totally different hardware. I suppose you could do that with hardware RAID but that’s vendor lock-in which I hate. It’s also nice to be able to use the same tools as your OS to deal with RAID rather than some foreign BIOS tie-in. I have used hardware RAID once or twice and felt it really slowed down booting. IBM x-series seems to do a SCSI scan with a very long timeout, something like 30s. Linux software RAID takes the spin-up time and a second or two. There’s nothing definitive but it all adds up to my preferring software RAID. Some say there is a definite advantage for RAID5 where everything has to be XORed. On AMD64, software RAID0 and 1 is mostly pointing drivers at buffers and DMA happens. I don’t know about ARMed SoCs but I can’t imagine the Cortex A57 CPU does the I/O. Some little building block does it just like a chip on a PCI bus.

  11. dougman says:

    Software RAID is a dead-end. Do not ask me how I know.

    I always used hardware raid now since then. In fact, I do not even use RAID now, just use a JBOD + Parity. However, this will be soon replaced by a dual-parity arrangement.

  12. Dr Loser says:

    Beast lives on!

    Can we assume, then, that you will not be investing your hard-earned money in that nasty Cello thing — a desideratum to which both Dougie and myself have adverted you against?

    I do so hope that you didn’t also stock up on that cheapo 2x8GB of DDR3 RAM that I pointed out was probably the extreme end of your wish-list for the Viola.

  13. dougman wrote, “What do you do about redundancy? Backups??”

    I prefer software RAID and I can put a SATA/USB dongle on one of the clients for backups. I’m going to use some combination of RAID0 for storage (faster transfers) but I could have two RAID0s in RAID1 for redundancy. I will have several times the storage of Beast either way and it will be much faster. I will experiment with the card on Beast for a while and try different combinations. The desired motherboards still are not offered outright for sale… They were promised for Q2 and this is the last day of Q2.

  14. dougman says:

    “Does not support Hardware RAID”

    What do you do about redundancy? Backups??

Leave a Reply