Samsung Caught Cheating in TV Benchmarks

The FPS Review may receive a commission if you purchase something after clicking a link in this article.

Image: Samsung

Samsung has been caught cheating in TV benchmarks, according to recent reports by FlatPanelsHD and HDTVTest’s Vincent Teoh that discuss and demonstrate how the company has designed its TVs to “recognize and react to test patterns used by reviewers.” Among the cheats that Samsung has seemingly engineered include those discovered in the QN95B Neo QLED 4K Smart TV, which reportedly “changes its color and luminance tracking during measurements to appear very accurate” and “boosts peak brightness momentarily by up to 80%, from approx. 1,300 nits to 2,300 nits.” FlatPanelsHD noted in its coverage that it had reviewed the set but found that it never surpassed 1,300 nits with real content. Samsung has promised a software update.

“Samsung remains committed to relentless innovation to provide the best picture quality to our consumers,” wrote in a statement to FlatpanelsHD. “To provide a more dynamic viewing experience for the consumers, Samsung will provide a software update that ensures consistent brightness of HDR contents across a wider range of window size beyond the industry standard.”

“The update for S95B has been conducted, and the update for QN95B will be provided soon,” the company added.

Reviewers, calibrators and certification bodies typically use a 10% window for HDR testing, which simply means that it takes up 10% of the screen. In this window multiple steps from black to white as well as a set of colors are measured. Samsung has designed its TVs to recognize this and other commonly used window sizes, after which the TV adjusts its picture output to make measurements appear more accurate than the picture really is. When using a non-standard window such as 9% (everything else equal), the cheating algorithm can be bypassed so the TV reveals its true colors.

Source: FlatPanelsHD

Join the discussion in The FPS Review Forums...

Discussion (19 replies)

Join Discussion →
Niner51
Niner51 👍 2

Brian_B
Brian_B 👍 1

I can’t believe this is even a thing

LazyGamer
LazyGamer 👍 1

So, quack.exe rises again!

And as I'd suspected that this applied to Samsung's LCD panels... but nope, suspected wrong, this is their brand-new QLED tech.

Back to lusting over a 77" LG panel for the living room :cautious:

MadMummy76
MadMummy76

"Brian_B, post: 54623, member: 96" wrote:

I can’t believe this is even a thing


I can totally believe this is a thing everywhere, too bad they only get caught occasionally.

D
Dan_D 👍 1

AMD, NVIDIA, ASUS and just about every other computer hardware maker has cheated on benchmarks at one time or another. I'm not surprised in the least.

Peter_Brosdahl
Peter_Brosdahl 👍 1

"LazyGamer, post: 54624, member: 1367" wrote:

So, quack.exe rises again!



And as I'd suspected that this applied to Samsung's LCD panels... but nope, suspected wrong, this is their brand-new QLED tech.



Back to lusting over a 77" LG panel for the living room :cautious:


As I'm sitting in front on my CRG9 I'm saying "there there baby, everything is going to be alright". Meanwhile, I can comfortably see past it to the old LG C9. ;)

I do agree with you about the text on it but I can't do justice to how great games look on it. 4K/120 Hz G-Sync/12-bit HDR. I was playing Metro Exodus on it the other day and I do take moments just to look around.

Riccochet
Riccochet

It's still the best LCD TV on the market. Just like their QD-OLED is the best on the market. While LG was forced to push even more power through their C2 OLED to compete.

h
hubaduba 👍 3

It does that to optimize battery usage.

Peter_Brosdahl
Peter_Brosdahl

"Riccochet, post: 54650, member: 4" wrote:

While LG was forced to push even more power through their C2 OLED to compete.


I was actually looking at the 42" version of that yesterday along with the Alienware 34" that uses the Samsung QD-OLED. Tough choice between the two. About the same price and has many similar features although the Alienware is 21:9, which I prefer. However, the C2 is brighter and I prefer their screens and the review from Hardware Unboxed had mixed things to say about the Alienware. Rtings gave it a good review though.

Riccochet
Riccochet

"Peter_Brosdahl, post: 54652, member: 87" wrote:

I was actually looking at the 42" version of that yesterday along with the Alienware 34" that uses the Samsung QD-OLED. Tough choice between the two. About the same price and has many similar features although the Alienware is 21:9, which I prefer. However, the C2 is brighter and I prefer their screens and the review from Hardware Unboxed had mixed things to say about the Alienware. Rtings gave it a good review though.


Look at the LTT review showing the LG C2 and Samsung QD-OLED panels side by side. I'll take the Samsung all day. Colors on the LG just did not look right. Like they were pumping more power to the greens and blues to compensate for lower brightness. Games on the Samsung just looked ridiculous, reds and yellows were vibrant, while they looked washed out on the LG.

And that's all LG is doing to get higher brightness. They're pumping more power through the panel. That's going to result in faster organic decay and burn in.

Brian_B
Brian_B 👍 1

"Riccochet, post: 54654, member: 4" wrote:

Look at the LTT review showing the LG C2 and Samsung QD-OLED panels side by side.


When you are making a subjective opinion, can you really trust visuals from a compressed stream to be accurate though?

All the objective data I’ve seen tends to point to the C2 as the better screen. Here is just one example:

[URL unfurl="true"]https://www.tomsguide.com/news/we-just-tested-samsungs-first-qd-oled-tv-and-the-results-will-surprise-you[/URL]

That said - numbers aren’t everything and screens are highly subjective: you like what you like.

Peter_Brosdahl
Peter_Brosdahl 👍 2

"Riccochet, post: 54654, member: 4" wrote:

Look at the LTT review showing the LG C2 and Samsung QD-OLED panels side by side.


I'll have to check that one out later. There did seem to be some disparity between Hardware Unboxed (HU) and Rtings so a third would be nice to see. HU said they had to calibrate theirs in order to get much better color accuracy while Rtings said they needed very little calibrating. I'm also concerned about its brightness as this is in a room that's fairly bright. Both mentioned how HDR tends to hang around 250-400 even though it is rated at 1000. Anything around 400 or less is just too dark.

I have a C9 and love it but at 65" it's the living room tv. For gaming its brightness has never really been an issue but for media consumption that's another story. Meanwhile, I also have the CRG9 which is just a tad too big at 49" but also doesn't really compare to the IQ I see on the C9. I'm also still on the fence in trying to decide if a 42" C2 is too big. For me, the sweet spot is around 38"-40".

Ultimately I'll probably wait for something else after Alienware/Samsung because the general consensus is that things will only get better as more models get released. I also found it a bit discerning the graph similarities between HU's testing and the ones I saw from Teoh for the other Samsung panels. Not a good time to be seeing such a thing. Also not happy that HU's review mentioned the cooling fan for the Alienware is fairly audible. I'm really not into having a display with a cooling fan on my desktop.

I finally got my main gaming rig set up the way I want but now back to the seemingly never-ending quest for a display that checks all the boxes I'm looking at. Ugh, I know it's first-world problems but so tired of looking. I do like that both of these are in the $1300-$1400 range which, while still high, is far better than the $2K-$3K other similar things are going for. It also seems like display manufacturers are all having some kind of QA issues when it comes to things in this tier bracket. I couldn't tell you how many reviews (MSI/ASUS ROG/Samsung/LG) I've read in the last 6 months of things I thought looked good only to find out about some quirk in the design that people found out about after buying them.

LazyGamer
LazyGamer 👍 2

"Peter_Brosdahl, post: 54664, member: 87" wrote:

I'm also concerned about its brightness as this is in a room that's fairly bright. Both mentioned how HDR tends to hang around 250-400 even though it is rated at 1000. Anything around 400 or less is just too dark.


Since I'm a vampire, I assume that I wouldn't be able to survive in said room...

"Peter_Brosdahl, post: 54664, member: 87" wrote:

It also seems like display manufacturers are all having some kind of QA issues when it comes to things in this tier bracket.


I'm a fan of Alienware for this reason, though the reasoning may not apply to readers outside of NA. Still, there's significant confidence here that I just don't have in most other vendors.

"Peter_Brosdahl, post: 54664, member: 87" wrote:

I couldn't tell you how many reviews (MSI/ASUS ROG/Samsung/LG) I've read in the last 6 months of things I thought looked good only to find out about some quirk in the design that people found out about after buying them.


I didn't think the VA panel in the 32" 1440p LG monitor I have could be that bad, with reviews basically saying that the text issues could be overcome and that the monitor had pretty good color.

I've had to run the monitor at 125% scaling to 'overcome' the text issues, and after years of trying, I've yet to be able to consistently calibrate it. Now, this isn't a Samsung "VA", but sourced from someone else, and perhaps that should have been the warning sign. It does have better contrast on static scenes than any IPS and doesn't look that bad, but aside from contrast an even older 27" 1440p IPS panel blows it out of the water.

"Riccochet, post: 54650, member: 4" wrote:

It's still the best LCD TV on the market. Just like their QD-OLED is the best on the market. While LG was forced to push even more power through their C2 OLED to compete.


I'd challenge that. Not for the purpose of completely refuting the claim, but rather to point out that the differences don't really seem to measure up one way or the other, and so 'best' will fall into the subjective spectrum.

"Peter_Brosdahl, post: 54652, member: 87" wrote:

I was actually looking at the 42" version of that yesterday along with the Alienware 34" that uses the Samsung QD-OLED. Tough choice between the two. About the same price and has many similar features although the Alienware is 21:9, which I prefer. However, the C2 is brighter and I prefer their screens and the review from Hardware Unboxed had mixed things to say about the Alienware. Rtings gave it a good review though.


So, I'm sticking with my AW3821UW with its IPS panel. 21:10, 38", sharp text, great color, great response times, and enough contrast.

As important as gaming is, so is being able to work, and I've found this size and aspect ratio to actually be optimal for both, for me.

Peter_Brosdahl
Peter_Brosdahl 👍 1

"LazyGamer, post: 54667, member: 1367" wrote:

So, I'm sticking with my AW3821UW with its IPS panel. 21:10, 38", sharp text, great color, great response times, and enough contrast.


I'll have to look into that one. I just pulled it up on the Dell website. Interesting. I was so hung up on 21:9/32:9 I hadn't even considered 21:10 and that actually makes a bit more sense. Even though I like the added width of those others I do miss some of the height. Still up there in price but at least gives me something else to think about. Thanks!

Peter_Brosdahl
Peter_Brosdahl 👍 2

"LazyGamer, post: 54667, member: 1367" wrote:

Since I'm a vampire, I assume that I wouldn't be able to survive in said room...


lol! I'm a man of extremes. I sometimes like it bright enough to get a tan(we do have skylights in the living room) but then during the heat of the summer, I've been known to reuse the cardboard packaging from our TVs to completely block out the windows in my cave for utter darkness. That desk I posted pics of in another thread is in another room where I've attached Velcro on one side of the curtains to keep them flush to the wall in order to prevent light bleed when I close them.

Tsing Mui
News poster at The FPS Review.

Recent News