VideoLAN has launched a new version of the VLC media player that includes support for RTX Video Super Resolution (VSR), NVIDIA’s new technology for upscaling lower-quality video with the help of AI and a deep learning network. VLC 3.0.19 RTX Vetinari is a special version of the “Vetinari” branch of the popular media player with RTX upscaling, VideoLAN has confirmed, and according to the change log, this version of VLC activates VSR upscaling by default on NVIDIA GeForce RTX GPUs that support the feature (i.e., GeForce RTX 30 and 40 Series). Some users claim that NVIDIA VSR is capable of upscaling video just as well as madVR, the hugely popular video renderer from madshi that allows users to choose from various high-quality upscalers, including the GPU-intensive NGU models, but based on some of the comparisons that have been shared online (1, 2), reception of the results are sure to vary from user to user.
From an NVIDIA post:
RTX VSR is a breakthrough in AI pixel processing that dramatically improves the quality of streamed video content beyond edge detection and feature sharpening.
Blocky compression artifacts are a persistent issue in streamed video. Whether the fault of the server, the client or the content itself, issues often become amplified with traditional upscaling, leaving a less pleasant visual experience for those watching streamed content.
RTX VSR reduces or eliminates artifacts caused by compressing video — such as blockiness, ringing artifacts around edges, washout of high-frequency details and banding on flat areas — while reducing lost textures. It also sharpens edges and details.
The technology uses a deep learning network that performs upscaling and compression artifact reduction in a single pass. The network analyzes the lower-resolution video frame and predicts the residual image at the target resolution. This residual image is then superimposed on top of a traditional upscaled image, correcting artifact errors and sharpening edges to match the output resolution.
The deep learning network is trained on a wide range of content with various compression levels. It learns about types of compression artifacts present in low-resolution or low-quality videos that are otherwise absent in uncompressed images as a reference for network training. Extensive visual evaluation is employed to ensure that the generated model is effective on nearly all real-world and gaming content.
Discussion (8 replies)
Join Discussion →I see the mention of deep learning networks in this article a lot. Does this require some kind of always on network connectivity to work?
I don't think so - pretty sure it's just using the DLSS algorithm on video.
If I remember correctly, DLSS uses the tensor cores on RTX cards for its AI deep learning processing.
You see I thought that Nvidia ran data through it's own AI farms to process images for best upscaling results and passed those methods through configuration for the local Tensor cores to handle processing of local game data. And if a game wasn't in the DLSS database it wasn't DLSS compatible. Hence why DLSS only works with specific games.
Perhaps that has changed from previous iterations.
BUT...
If VLC is doing it... I think it would be based on content flags of some sort or color saturation of the image to determine the best DLSS method to upscale the image and preserve clarity/detail. I don't see Nvidia running ALL video content through it's farms to teach based on... metadata tags... if they exist. Other than perhaps some new tag that is generated on the new version of VLC. And I wouldn't expect the results to be as good... I mean I would think FSR upscaling in this case might be on par.
I think that was true with DLSS 1.0. But it kinda sucked, because it only worked at specific resolutions and ratios that the game happened to be trained on. So 2.0 came out and was a more generic "AI-based upscaler" -- I don't think it really uses any AI that requires training per-game anymore, but does somehow use Tensor cores - so maybe it's a generic AI-assisted upscaler? Maybe they just left AI in the name so it sounds cooler and gives a good excuse to use Tensor cores? Maybe it's really Skynet? I don't know.
https://en.wikipedia.org/wiki/Deep_learning_super_sampling#:~:text=In 2019, the video game,not use the Tensor Cores.
So currently it's the "It does use Tensor Cores" version and "It's local"
Some of these questions are answered in NVIDIA's FAQ: https://nvidia.custhelp.com/app/answers/detail/a_id/5448/~/rtx-video-super-resolution-faq
Also see the external links from Wikipedia's (mostly empty at this time) article: https://en.wikipedia.org/wiki/Video_Super_Resolution
[embedded media]
Doesn't appear to be available on Linux (from what info I've come across so far), and locked to browsers I don't use, so no advantage to me. On the other hand, doesn't seem like I'm missing out on much either. Cool that VLC has it now, and again I wonder if that includes Linux.
The "VSR" initials confuse me, cuz didn't Radeons have tech called VSR that was their equivalent to nVidia's DSR (you know, running a game at a resolution higher than what your display supports, then downscaling it to your display's native res)? So now there's an nVidia VSR and an AMD VSR.
Ah yeah, see, this shiznit: https://www.amd.com/en/technologies/vsr
Which was the AMD equivalent of this: https://www.nvidia.com/en-us/geforce/technologies/dsr/technology/
"VSR" was already taken nVidia, You should have chosen something else.