VAT & Duty Inclusive

Benchmark Cheating is Back, and there's Hell to Pay

If you’re thinking of ditching your old iPhone for a cheaper (and presumably faster) OnePlus phone, don’t.

A few days ago, OnePlus and a string of other smartphone OEMs were caught benchmark cheating by XDA Developers, a famous hub for developers and user-generated smartphone reviews. XDA revealed that they found that the OnePlus 3T reports inaccurate performance values when tested with certain benchmarking apps.

When run with apps like Geekbench, the OnePlus 3T begins a trigger process that reports processor clock values to the app that are significantly better than they actually are—even when the processor was supposed to idle. With the help of Geekbench creator, Primate Labs, XDA tested the OnePlus 3T with a version of Geekbench 4 “disguised” as an app called “Bob’s Mini Golf Putt.” In addition to the inflated values, they also found that the feature had been much a part of Hydrogen OS phones (marketed in China) for a long time, and was only recently added to the Oxygen OS for Nougat builds.

This set off a series of benchmark tests on other phones, which found the Meizu Pro 6 guilty of the same deception. In a bid to sell that the phone can alternate between high-power and efficiency modes, it inflated scores while on “Performance Mode” when tested with popular benchmarking apps, when it actually performed the same when on “Balanced Mode”. A few other currently unnamed smartphones have since been found to do the same.

This was not the first time OEMs have cheated on their benchmarks. Back in 2013, benchmark cheating was found to be a staple across the smartphone range, which came to a head with significant backlash from angry users. Only Apple and Motorola were found exempt from cheating, while Samsung, HTC, and other major manufacturers were all found guilty.

When reached for comment, OnePlus quickly gave a response promising that the trigger process would no longer be present in future updates and builds:     

In order to give users a better user experience in resource intensive apps and games, especially graphically intensive ones, we implemented certain mechanisms in the community and Nougat builds to trigger the processor to run more aggressively. The trigger process for benchmarking apps will not be present in upcoming OxygenOS builds on the OnePlus 3 and OnePlus 3T.’

In all fairness to OnePlus, the OnePlus 3T still is a good phone. If anything, this incident is telling of how far some OEMs will go to look good on paper. However, many depend on benchmarks when looking for prospective smartphone buys, and benchmark cheating ensures that users don't get what they pay for.

Steven Zimmerman, who broke the news to the XDA community, cautioned readers to always be on the lookout for benchmark cheating.

'Unfortunately, the only real answer to this type of deceit is constant vigilance. As the smartphone enthusiast community, we need to keep our eyes out for attempts to deceive users like this…  Issues like this one can be added days, weeks, months, or even years after the device launches, artificially inflating the global averages gathered by benchmarks months down the line, influencing the final database result.'

Our advice? Buy an iPhone, or if you have one, stick to it. Because with Apple, you always get exactly what you signed up for.

Visit our Facebook page here for more exciting updates, and visit our site here to get the most amazing Refurbished iPhone deals you’ll find online!

  • Feb 03, 2017
  • Category: Articles
  • Comments: 0
Leave a comment