There has always been a debate about the relevance of benchmark tests in the world of smartphones, and whether users really should bother with the so-called test scores that tend to keep floating around the World Wide Web and social media from time to time. Sometimes, even phone makers use these numbers to advertise the supposed performance superiority of their new or upcoming phones. A week ago, technology website AnandTech published a detailed technical report that revealed how MediaTek was clearly cheating the benchmark tests on mobile phones. A subsequent statement issued on the matter by MediaTek does not deny it at all, but instead attempts to justify and explain why it has been doing so, and why it thinks that it's okay to do so. The company's attempt to cover up negative media coverage actually sparked off a chain of events that also saw its competitors get dragged into the matter. All of this has had technology reviewers and commentators in the industry opine on the matter, even though technology coverage as we know it has been slow in light of the ongoing coronavirus pandemic. However, as a user, should this really make any significant difference to you?
One thing about benchmarks
The answer is actually a bit complicated. Straight off the bat, those of us who have covered technology (devices, to be specific) have known for years that benchmarks are not quite the gold standard of representation of what a smartphone is capable of. Further, with time, processors have improved in leaps and bounds, and in today's market, you may actually be hard pressed to find a smartphone that can be legitimately called 'bad'. Comparatively, sure, but not in the outright, laggy fashion that we've all seen in budget Android devices from a few years ago.
In such a market, the hardcore relevance of a benchmark test remains debatable. That said, it is still an important metric, because the premise of a benchmark is to give devices a uniform testing field, and produce a number that conveys which device performs better than the other. Historically, benchmarks have seldom been the correct representation of what a smartphone is capable of doing in real life. Yet, it does attempt to show some data in terms of what a user can expect from a phone, before buying it. Most such tests are designed to test devices in medium or at least high intensity tasks, which the average user may often not do on their phone. For example, a Qualcomm Snapdragon 7xx processor has plenty of firepower for a user to not even break a sweat when running heavy games, making benchmarking scores moot to the average consumer.
A company that 'lied'
The problem here, more than how MediaTek's usage of a high performance mode to cheat benchmark performance, is the fact that the world's second most recognised non-proprietary chipset maker found it okay to create fabricated numbers for tests that are designed to win consumer trust. In every way, this is an anti-consumer behaviour, and to make matters significantly worse, MediaTek does not even seem to be too bothered about it. For reference, here are a few key excerpts of the statement that MediaTek furnished to Andrei Frumusanu's report as an explanation:
In other words, MediaTek's statement essentially states that it views benchmarks as a way of advertising what its products are capable of doing when ideal conditions persist. Remember old physics laboratory experiments, where we assumed surface friction to be zero for the sake of a theoretical problem? The explanation offered is similar in nature. In a typical smartphone chipset, the high performance cores are only enabled for short spurts when a user plays an intensive game, and that is not how the smartphone would perform in most other conditions. Furthermore, by stating that OEMs implement similar practice as well (a known but criticised issue), MediaTek's statement is a clear abrogation of consumer faith.
A statement shared by a Qualcomm spokesperson with News18 says, "Whitelisting refers to the technique of using the app name to determine whether to put the device into performance enhancement mode. The action of whitelisting a benchmark app is generally considered by the industry as cheating since it defeats the purpose of a benchmark, which is to reflect user experience for day to day use. Qualcomm does not whitelist." Qualcomm is the world's largest maker of smartphone processors, according to IHS Markit data from the last quarter of 2019.
The question would then be, why should MediaTek really adopt such practices, if an industry competitor manages to make its devices perform well and yet does not need to paint a fabricated picture via these very same benchmarks?
Should you really care?
Now comes the issue of you, the consumer. In terms of its direct impact on a consumer, nothing will likely change about your phone. Devices such as the Redmi Note 8 Pro (running on a MediaTek Helio G90T) and the Oppo Reno 3 Pro (running on Helio P95) would perform pretty much the same, and if you are happy with how your phone is performing, there's no need to worry as nothing is going to change about it. However, what you do need to know, is that if you were enamoured by their performance scores flaunted upon launch, you were most likely duped.
Comments
0 comment