NVIDIA Frame Generation was one of the new features announced with DLSS 3 that can fill in frames for games to boost overall framerates for a smoother gaming experience. NVIDIA Frame Generation uses Optical Flow Acceleration (OFA) to do calculations with Tensor Cores. Brent Justice explained this in his NVIDIA GeForce RTX 4090 Founders Edition Review which can be found here.
From NVIDIA DLSS 3 Press Release:
“Optical Multi Frame Generation generates entirely new frames, rather than just pixels, delivering astounding performance boosts. The new Optical Flow Accelerator incorporated into the NVIDIA Ada Lovelace architecture analyzes two sequential in-game images and calculates motion vector data for objects and elements that appear in the frame, but are not modeled by traditional game engine motion vectors. This dramatically reduces visual anomalies when AI renders elements such as particles, reflections, shadows, and lighting.
Pairs of super-resolution frames from the game, along with both engine and optical flow motion vectors, are then fed into a convolutional neural network that analyzes the data and automatically generates an additional frame for each game-rendered frame — a first for real-time game rendering. Combining the DLSS-generated frames with the DLSS super-resolution frames enables DLSS 3 to reconstruct seven-eighths of the displayed pixels with AI, boosting frame rates by up to 4x compared to without DLSS.“
DLSS 3 arrived with the launch of the GeForce RTX 4090 graphics card and is still in its early adoption phase, but it has already been integrated into Marvel’s Spider-Man: Remastered. Igor Wallosek (Igor’s Lab) has discovered while testing DLSS 3 that NVIDIA Frame Generation can be turned on while using other supersampling methods from AMD and Intel. He contacted NVIDIA about this and was told that it was intentional but the results may not be as good as those seen when using DLSS.
“We asked NVIDIA if this was intentional with the settings menus and got the confident answer that it was a feature and not an oversight, while they also pointed out that DLSS would provide the best picture quality, but that the frame generation would in principle also work with all other methods like AMD’s FSR and Intel’s XeSS.”
FSR 2.1 and Intel XeSS testing results
Igor’s testing did in fact yield interesting results. He tested using the aforementioned GeForce RTX 4090 in 4K. Frames did in fact boost to impressive levels. Using FSR 2.1 (Ultra Performance), with ray tracing (DXR) set to very high, frames were boosted by nearly 100 FPS. They went from 135.6 (frame generation off) to 231.4 (frame generation on). Now the Ultra Performance mode is usually not a preferred setting for those wanting the best image quality but even when on the quality setting there were still significant gains seen going from 127.8 FPS to 193.8 FPS. Intel XeSS yielded FPS gains as well. In Performance mode, FPS topped at 203.9 FPS(on) vs 122.8 FPS (off). The gap was a bit smaller at Ultra Quality with 168.5 FPS (on) vs 130.2 FPS (off) but that is still over 30 FPS gained. However, Igor did state that the Ultra Quality setting did not bring any meaningful gain over the native render (less than 1 FPS).