Benchmarking has been apart of testing computer hardware for decades, but we all know that it can sometimes not be the very best way of determining which devices are more powerful or usable than others. Today, that truth has been even further proven with AnandTech reporting that Samsung’s Galaxy S4 may not be the most honest of smartphones when being put through various Android benchmarking apps.
It all started at the Beyond3D forum where a user claimed that Samsung has employed some tricks to make its flagship smartphone perform better when being benchmarked. Basically, the post claimed the GPU of the Exynos 5 Octa Galaxy S4 has been set to run at higher frequency when it detects that a benchmarking app is being run. AnandTech decided to investigate and, according to their tests, the device’s graphics processing unit runs at 533MHz during benchmarks, while running at a slightly lower 480MHz during normal use and within other games and apps.
The CPU speed of the device also seems to be getting the same devious treatment. While running GLBenchmark 2.5.1, AnTuTu, Linpack, and other benchmarking Android apps, this variant of the Galaxy S4 seemed to whirl the Cortex A15 cores at 1.2GHz. But when running GFXBench 2, which supposedly isn’t affected by Samsung’s modifications, the device ran at its standard 500MHz speed. “It would appear that only GLB2.5.1 is allowed to run in this higher performance mode,” says Brian Klug. That’s a pretty stark difference.
So what’s the culprit behind these juiced up speeds? AnandTech points out some code labeled “BenchmarkBooster” which seemingly makes sure the device performs at its absolute best when being tortured by the Android benchmarking apps listed above.
What this does mean however is that you should be careful about comparing Exynos 5 Octa based Galaxy S 4s using any of the affected benchmarks to other devices and drawing conclusions based on that. This seems to be purely an optimization to produce repeatable (and high) results in CPU tests, and deliver the highest possible GPU performance benchmarks.
We’ve said for years now that the mobile revolution has/will mirror the PC industry, and thus it’s no surprise to see optimizations like this employed. Just because we’ve seen things like this happen in the past however doesn’t mean they should happen now.
In the end, it’s simply disappointing that Samsung even has to do this. As we’ve seen with the iPhone, it’s very possible to achieve a friendly and lag-free user experience with even not-the-most-powerful hardware. Basing your smartphone purchases on benchmark results could very well lead you astray, because being able to destroy benchmarks doesn’t necessarily mean you’re going to end up with the best experience. And this news makes it seem like that may be more true now than ever. AnandTech thinks Samsung should either open up the ability for this hardware boost for everyone, or remove the optimization altogether. What do you think?