💾 Archived View for thfr.info › openbsd › Compile-Time-Comparison-Oct-2021 captured on 2023-12-28 at 15:20:35. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-01-29)
-=-=-=-=-=-=-
Article Update Information:
I recently bought the Asus ExpertBook B9400CEA. Buying newer hardware when using an operating system with a limited number of kernel developers like OpenBSD is always a bit of a gamble when it comes to how much will work as expected. This laptop looked like it was worth it. It seems to hit the sweet spot between portability, battery life, performance, and customizability (in my case especially multibooting with 2 separate M.2 SSD drives).
See my initial setup and impressions here
In terms of performance, I was expecting to come at least reasonably close to the performance of my desktop i7-10700 CPU. The i7-1165G7 in the ExpertBook only powers 4 cores and is a chip designed for ultra-portable, low-power laptops (Evo/Ultrabook family). However, the newer 10mm Tiger Lake architecture, the high peak CPU speeds (4.7GHz), and the decent scores on CPU benchmarks suggest this one is not a total slouch.
PassMark assessment and comparison of the i7-1165G7
After my initial rather positive impressions with the new Asus ExpertBook B9400CEA, I now have a bit more experience on it and some of the more subtle observations led me to this article. Specifically, I noticed unexpected issues with performance in a couple of situations:
All of this together is why I set out to try a more objective way to compare the performace with other OpenBSD machines that I have around.
Here is what I have available, as a line-up of OpenBSD machines that I've used in the last ~5 years. They are all amd64 architecture.
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |Manufacturer Name CPU Cores RAM HDD Year (CPU) PassMark score | +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ |<self-assembled desktop> N/A Intel Core i7-10700 (2.9-4.8 GHz) 8 32G DDR4 PC4-17000 WDC WDS200T2B0B 2020 17,213 | |Asus ExpertBook B9400CEA Intel Core i7-1165G7 (2.8-4.7 GHz) 4 16G LPDDR4X on board ADATA Technology SX8200 Pro 2020 10,669 | |Lenovo ThinkPad X395 AMD Ryzen 7 PRO 3700U (2.3-4.0GHz) 4 8G DDR4 RAM WD NVME drive 2019 7,311 | |Dell Latitude E5570 Intel Core i7-6820HQ (2.7-3.6 GHz) 4 16G DDR4 PC4-19200 some NVME HDD 2015 6,922 | +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I have been using the desktop i7-10700 for much of 2020 to 2021 and it has been a fairly reliable OpenBSD work horse. When I bought the ExpertBook with the i7-1165G7, I was hoping for a mild to moderate improvement in graphical/gaming performance, and at least getting close to the raw CPU performance of the i7-10700.
AMD Ryzen CPUs have been snatching market dominance from Intel over the last years, so I was expecting a fairly decent result from the Ryzen 7 PRO 3700U. It was hard to place based on gut feeling. Let's say, I wouldn't have been surprised if this one ended up outperforming most other CPUs.
Finally, the Latitude E5570 is by far the oldest machine in this collection. At the time, it was one of the beefiest laptop CPUs I could find at an affordable price, but that was about 5 years ago and I bought it used at the time. In addition, what I picked up from discussion around the mitigations for Intel's Spectre and Meltdown-related issues seemed to indicate that Skylake CPUs (like the i7-6820HQ) would be the most affected in their performance because of the problems with the architecture.
I've made a few preparations to try to ensure a comparison that's as even as feasible:
Here is how I went about my testing:
$ cd /usr/ports/games/godot $ make clean patch $ MAKE_JOBS=4 time make && uptime
Note that the first run on the i7-10700 was done with all 8 MAKE_JOBS (and is unsurprisingly the fastest result).
+-----------------------------------------------------------------------------------------------+ |CPU MAKE_JOBS Compile Time (games/godot) Load Average | +-----------------------------------------------------------------------------------------------+ |Intel Core i7-10700 (2.9-4.8GHz) 8 7m14 7.64, 7.02, 3.81 | |Intel Core i7-1165G7 (2.8-4.7GHz) 4 32m06 3.93, 4.76, 4.38 | |AMD Ryzen 7 PRO 3700U (2.3-4.0GHz) 4 30m01 3.47, 4.62, 4.24 | |Intel Core i7-6820HQ (2.7-3.6GHz) 4 18m38 3.65, 4.58, 3.52 | +-----------------------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------------------+ |CPU MAKE_JOBS Compile Time (games/godot) Load Average | +-----------------------------------------------------------------------------------------------+ |Intel Core i7-10700 (2.9-4.8GHz) 4 11m11 4.18, 4.13, 2.92 | |Intel Core i7-1165G7 (2.8-4.7GHz) 4 33m27 3.57, 4.71, 4.67 | |AMD Ryzen 7 PRO 3700U (2.3-4.0GHz) 4 30m17 3.42, 4.62, 4.52 | |Intel Core i7-6820HQ (2.7-3.6GHz) 4 19m01 3.65, 4.60, 4.06 | +-----------------------------------------------------------------------------------------------+
And as a visual representation of the second run where all ran with MAKE_JOBS=4 (shorter is better):
Intel Core i7-10700 (2.9-4.8GHz) ########### Intel Core i7-1165G7 (2.8-4.7GHz) ################################# AMD Ryzen 7 PRO 3700U (2.3-4.0GHz) ############################## Intel Core i7-6820HQ (2.7-3.6GHz) ###################
It was my gut feeling that something about the performance of the ExpertBook's i7-1165G7 wasn't meeting expectations, and this was the initial incentive to do this comparison testing. The overall result of this is surprising:
Note that for someone who spends quite a bit of time on ports, the magnitude of the difference is striking. This means that when using all cores, it takes MORE THAN 4 TIMES AS LONG to complete compilation on the i7-1165G7, compared to the i7-10700. Even when leveling the playing field by using the same number of MAKE_JOBS, it still takes 3 times as long. This means less productivity and longer wait times.
In addition, as I use OpenBSD heavily as my daily driver even while compiling, for example by browsing the internet, sometimes with many browser tabs open, reading, watching videos, etc., the poor performance also limits usability during compilation. Casually browsing a few web pages while letting the compiler do its work is rather painful on the i7-1165G7, as there is lots of stuttering and lag.
On top of that, even in normal, relatively limited use scenarios like _just_ using the browser (without concurrent compute-intensive tasks) or playing a game, there is a noticeable slowness to the system.
Of course, this is far from a rigorous assessment of objective performance. There are some differences in other components that may affect the results, like size and type of RAM (note no swapping or memory exhaustion was noted when the process was monitored with htop), I/O speed of the disk, etc. Between several runs of compiling games/godot, the compile time was quite consistent and differed by less than 1 minute in most cases (note the i7-10700 was initially run with 8 MAKE_JOBS).
Overall, the components like mostly NVME drives and the size and type of RAM make me think that it is unlikely that those had a significant influence on the results.
Instead, I would raise the following questions:
I wonder if in the current state of OpenBSD, the performance (with a multiprocess kernel) is very much dependent on the multi-threaded performance, as opposed to single-core performance. This could partially explain the results, as the performance testing of the i7-1165G7 suggests great single-threaded performance, but relatively underwhelming results in multithreaded loads.
In the end, the difference in performance is so drastic that I've decided that for now I will go back to doing my main work on the desktop, which runs the smoothest and fastest. I will probably use the Skylake i7-6820HQ for a while longer, given the surprisingly good results. I am curious if performance of the Tiger Lake CPUs will see some more improvements. This may be a hard problem to diagnose/solve - I can't tell how much (if any) of the performance issues are CPU-related, or if the thermals are a bigger factor. I hope to test the laptop again later on, maybe in half a year or so, to see if there have been more improvements. For now, this laptop is back to just running Windows when I need it (not that often..., I hope).