A somewhat bizarre and unexpected drama has been unfolding over the last couple of weeks between Linus Sebastian (LTT/LMG) and Steve Burke at Gamers Nexus. The two individuals and their respective sites have been well-known PC hardware reviewers for many years now. However, an LTT/LMG employee was recorded during a recent tour commenting on how its reviews differ from other sites like Gamers Nexus and Hardware Unboxed by using fresh benchmark data for every review instead of using older GPU data. This, in turn, prompted a response from Gamers Nexus who posted a 44-minute video unveiling a number of grievances with LTT but also expanding onto another, still ongoing, situation with a prototype GPU cooling solution from Billet Labs.
Per Gamers Nexus (via YouTube):
“This video is not monetized. This video covers our serious concerns regarding the data accuracy of Linus Media Group, including Linus Tech Tips, ShortCircuit, and TechQuickie, particularly as it relates to rushing content out the door to favor — by staff’s own admission — quantity over quality. As the company continues to expand into its LTT Labs direction, the importance of accurate data increases; however, even as ‘only’ entertainment, there are still certain responsibilities to the consumer and the manufacturers to report fairly (and to have defined corrections processes in place). We tried to approach this as objectively as possible and hope that viewers are able to listen to the evidence we present, particularly as it relates to significant and frequent data errors that now present in nearly every technical review video.”
In the above video, Steve breaks down a number of topics ranging from issues regarding reviews being rushed out too quickly as a quantity vs quality model seems to be happening, bad data, review errors for CPUs, GPUs, and PSUs, potential ethical concerns on numerous levels including the aforementioned prototype cooler from Billet Labs.
Review of Billet Labs cooler review goes completely off the tracks
If everything so far didn’t sound strange then the following goes right into full-on bizarre territory. A two-man-created company called Billet Labs which focuses on reinventing PC building submitted a prototype cooler for a GeForce RTX 3090 Ti. An LTT/LMG employee provided Linus with a GPU off the shelf for testing. At that moment, as caught on video, it was discovered that the GPU was actually a mislabeled RTX 4090, and Linus was obviously frustrated to find out they had one just sitting on the shelf but the team proceeded to conduct the review using the cooler with a GPU it was not designed for. As could be expected due to the manufacturing difference between the two GPUs, things did not go well for the cooler which resulted in a negative review of the cooler.
This was only the beginning of the drama for prototype as more was to come. At some point it was decided by LTT/LMG that it would be put up for auction but according to Billet Labs that was never agreed to by them. Additionally, there was no communication between the two regarding this until two to three hours after the above video from Gamers Nexus had been posted. Billet Labs thanked Steve for doing so and has since had some ongoing communications with Linus but those too are a bit convoluted, but Steve has vetted a timeline for them in another video response. He explains that the two have come to an agreement but in looking at the timeline it was a messy affair.
Evidently, the prototype had eventually been sold in a silent auction. LTT/LMG reportedly agreed to a “quote” value to reimburse Billet Labs but according to a response to GN, at the time on the matter, it said they never provided a quote only an estimated value. Billet has said that as of August 14, they have received another reply from Linus but have not yet responded back. They also go on to express gratitude to the entire community for its support during this stressful time.
Official Statement from Billet Labs (per Reddit):
“You, the PC community, are amazing. We’d like to thank you for your support, it means more than you can imagine.
Steve at Gamers Nexus has publicly shown his integrity, at the huge risk of backlash, and we have nothing but respect for him for how he’s handled himself, both publicly and when speaking directly to us.
…
Regarding LTT, we are simply going to state the relevant facts:
On 10th August, we were told by LTT via email that the block had been sold at auction. There was no apology.
We replied on 10th August within 30 minutes, telling LTT that this wasn’t okay, and that this was a £XXXX prototype, and we asked if they planned to reimburse us at all.
We received no reply and no offer of payment until 2 hours after the Gamers Nexus video went live on 14th August, at which point Linus himself emailed us directly.
The exact monetary value of the prototype was offered as reimbursement. We have not received, nor have we asked for any other form of compensation.
…
About the future of Billet Labs: We don’t plan to mourn our missing block, we’re already hard at work making another one to use for PC case development, as well as other media and marketing opportunities. Yes it sucks that the prototype has gone, it’s slowed us but has absolutely not stopped us. We have pre-orders for it, and plan to push ahead with our first production run as soon as we can.
We also have some exciting new products on our website that are available to buy now – we thank everyone who has bought them so far, and we can’t wait to see what you do with them.
We’re happy to answer any questions, but we won’t be commenting on LTT or the specifics of the email exchanges – we’re going to concentrate on making cool stuff, and innovative products (the Monoblock being just one of these).
…
We hope LTT implements the necessary changes to stop a situation like this happening again.
Peace out ✌
Felix and Dean
Billet Labs”
Linus Tech Tips posted its own response on its own forum which can be found here. It is far more lengthy and the following is only a small excerpt from it.
“There won’t be a big WAN Show segment about this or anything. Most of what I have to say, I’ve already said, and I’ve done so privately.
To Steve, I expressed my disappointment that he didn’t go through proper journalistic practices in creating this piece. He has my email and number (along with numerous other members of our team) and could have asked me for context that may have proven to be valuable (like the fact that we didn’t ‘sell’ the monoblock, but rather auctioned it for charity due to a miscommunication… AND the fact that while we haven’t sent payment yet, we have already agreed to compensate Billet Labs for the cost of their prototype). There are other issues, but I’ve told him that I won’t be drawn into a public sniping match over this and that I’ll be continuing to move forward in good faith as part of ‘Team Media’. When/if he’s ready to do so again I’ll be ready.”
Gamers Nexus on the defense
Steve does not hold back in defending Gamers Nexus’ approach to this, and other situations involving patterned behavior by corporations or individuals that cause harm to consumers. He emphasizes great disappointment in Linus’ numerous responses online, in LTT’s forums and it WAN shows, how Linus has dealt with these situations. Steve explains that Gamers Nexus does not normally reach out to corporations or individuals when damage has been done actively to consumers when it begins investigating such matters. This is due to said parties using prepared stating they are the victims, not the consumers.
There are many quotes and replies shown in the above videos, as well as in the LTT forums so those interested in more should check them out. An interesting side note is that this is all happening mere months after Linus Sebastian (LTT/LMG) said that he would be stepping down as CEO.
Discussion (19 replies)
Join Discussion →A lot of people especially Linus apologists are saying it's just drama for clicks, but I vehemently disagree. I'm really glad GN made that video, because it is great consumer advice: Do not trust LTT.
I understand not wanting to pick sides, but it is clear who is in the wrong here. This is a systemic issue for LTT, not just that they made a mistake. It is a policy of negligence and arrogance. Any credibility LTT had has just been utterly nullified.
Not because they made mistakes, everybody can, but their handling of said mistakes. Instead of apologizing and accepting blame they doubled down on their mistakes on multiple occasions. And they did not remove videos with egregious mistakes that could and possibly did mislead people, because they didn't want to loose any ad revenue.
There is no way back from here for Linus, he clearly stated he doesn't want to commit a few extra man hours to make sure what they show is good consumer advice. So there is no reason to take anything LTT publishes seriously. And even if they wow to change I wouldn't believe it, a zebra can't change its stripes.
There's a lot of tech tubers doing reaction videos to the news, where's "The FPS Review Reacts" video?? Gotta pump up those views!
Yea I have to agree. With all of the talk about LTT labs allowing super testing this is the result. And Linus with his kind of BS response was poorly delivered.
LTT needs to just own that mistakes wer made and with the rollout of LTT labs their goal is to be a trusted source. To that regard they will review policies and checks before publishing any data as they break new ground for LTT to have a labs and a entertainment division.
This is something the new CEO needs to get in front of and not Linus. Linus is a technically minded personality. He is not the most technically proficient resource and hasn't been for some time. I enjoy his enthusiasm and personality but some of his technical presentations especially as of late are cringe worthy.
We have truly moved into a new age. Shall we now say that America, perhaps the world, is living a Trumptopia, where information is a weapon and gas-lighting is expected.
I've old enough to remember the 60/70's when media sources were formal, apologized and worried about image. Not that those were perfect day, they wernt. Be so nifty if we could have the best of every decade.
Lol, I really wouldn't have anything to add to the conversation that hasn't already been stated. I think the best way to handle it, actually, is to talk more about our own methods for testing and ensure we are doing the best job with benchmarking. I will always continue to strive to do so.
I think they were being sarcastic. I would prefer you guys stay out of that mess. Even though the name of the site is thefpsreview I feel like things here are more professional and cordial. Even when madmummy and I tend to get snarky.
I look forward to content here. Your role as a provider of tech info is not to review other review sites. That's where I think gamers nexus went wrong. They should have just submitted concerns and cross posted them. But LTT threw down the gauntlet calling out gamers nexus and gamers nexus decided to pull a FAFO. Basically this all boils down to those in glass houses throwing stones.
If you want to be a source of good or great data for your target audience owning that is what you do. Getting embroiled in a pissing match between YouTube channels is stupid no matter who you are. While it's entertaining to observe from the outside. And we can commentate on the action on the field. It's a mess. It's distracting from what the channels are actually about. And it's doing little more than making LTT for the most part look like disengenuious fools. But it isn't being kind to Gamers Nexus either.
To add to the discussion, Hardware Unboxed has a lengthy podcast about this subject, and they mainly talk about benchmarking practices, it's a good watch, I watched the whole thing.
[embedded media]
I think it's very important to comment on this to the effect of how things are handled by each outlet. LMG trying to be fully automated without what appears to be almost any oversight was a giant red flag already and then Linus's attitude towards a product he already decided was shit before 'reviewing' and then fucking over that company and possibly giving that prototype to a competitor after promising to return it for a month.
LTT's response video was decent but they could have gone without the fake humor in it. This isn't a funny situation.
Yep, I was hard-pressed for time with this one and had a few more mistakes than I normally might do but tried very hard to stay neutral in presenting the facts. I may not have been totally neutral but I aimed for that. It's a mess to be sure and I doubt that it's over yet.
This SHOULD be a flash in the pan. Something to say... this helped ALL review channels/sites review their data and make sure that they are properly testing and vetting the data that they deliver to customers. AND the whole practice of super rushed reviews should be MARKETED as super rushed with potentially inaccurate data due to lack of quality controls on the distributed data.
What I mean is.... Pick your channel: "Hi guys gals and others, the embargo has lifted on X company's new X. We have put together review data and promise we have done the best we can in the window of time we have been allotted. We are doing a deeper dive on this product with what we consider a more complete review coming! We will do this as long as we feel we can improve the quality of a review with a deeper dive post embargo lift date. If you find any issues with the data please let me know! ON TO THE REVIEW!"
I would like to speak on the topic of automation of benchmarking.
Automation of benchmarking, naturally, can make things more efficient, speed up the process, and allow (at least in my opinion) to include more comparison data, which is always welcomed. The current method of manually doing everything reduces the comparisons one can simply do in each review, especially for launches.
That said, there are very serious inherent dangers in automation. The first is, settings can be set wrong by mistake, this very thing is what happened in a recent Linus review where a wrong setting in Cyberpunk (because of a patch change) caused his data to be simply wrong. Patches of games happen, very frequently, and sometimes how settings are applied, and what settings are applied, changes, and automation cannot keep up with this and DOES make mistakes.
The other danger of automation is simply being removed from the process itself, and missing problems and issues that arise from actual gameplay. If you are not playing the game, you are missing issues such as texture load-in, geometry, or detail load-in due to VRAM constraints. You are also missing the BIG picture of frametime and frame pacing smoothness in games. By removing yourself from the data collection, you are inherently removing yourself from the gameplay experience, and you are no longer relating the gameplay experience to the end user. The question then becomes, how relevant and informative is your review really?
I will emphasize, there are very big pro's for automation, very big, but also very big cons. I think both need to be understood and realized. Whatever method any outlet employs, should be 100% transparent to the reader in how that data is collected.
This is the response that they should have led with:
[embedded media]
I'm glad they got here in very short order.
All this 1,000 f*cking percent.
I think it's fixed by now but their apology video posted the unredacted $$ amount the block that Billet labs didn't want posted publicly. So they rushed that too. ?