💾 Archived View for gem.librehacker.com › gemlog › starlog › 20230921-0.gmi captured on 2024-06-20 at 12:09:45. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-09-28)
-=-=-=-=-=-=-
At my work, I've had a lot of projects to do this last year that involved reading and writing whole-disk drive images. This has given me more of an interest in the subject of read and write speeds, and I/O bottlenecks. Recently, I tried to optimize an image writing scenario, where I was writing an image to an SSD drive, using a USB<->SATA device adapter. My computer is a little old, but it has USB 3.0 ports. So I doubled-checked to ensure that I was using a USB SS port, and also that my adapter is at least 3.0 technology. USB 3.0, in my understanding, is supposed to get upwards of 500 MB/s speeds, and my adapter advertises to be a 6 Gb/s device. And the SSD drive, while not an expensive one, was claiming something like 360 MB/s write speed. So it seemed reasonable to expect a few hundred MB/s write speed.
But in practice, I was only getting around 10 or 20 MB/s. I was using dd to write the image, so I bumped up the block size (bs) to 16k, thinking maybe it was the I/O calls. The result was that for a minute or so, I got a really high throughput, about 200 MB/s, and then it dropped back down to 30 MB/s, and stayed at 30 MB/s for the remainer of the session.
It is my understanding, from online research, that the problem is the SSD drives themselves are based on flash memory technology and are inherently slow at writing. They just seem fast, in typical everyday use, because they come with a fast RAM cache that can take some data very quickly, end the I/O session, and then slowly copy the data into regular persistent memory while you are busy doing something else. But this system breaks down if you just keep sending data, and the cache fills up.
I've recommended to a lot of people that they use SSD drives, which I suppose is fine for most cases. But what if you actually want a high sustained write speed? Which direction to go? Are there some expensive SSD drives out there which use a faster write technology? Or switch back to mechanical drives? Or a hybrid SSD? Or something else I haven't heard of...? I'd appreciate any feedback from more knowledgeable geminauts.