Disney Research Studios has published a paper and video that it has prepared for the upcoming SIGGRAPH conference happening December 6-9. In its presentation, Disney demonstrates how a tool called FRAN is a production-ready solution for re-aging actors. Disney has already used re-aging in many of its projects spanning from Tron: Legacy (2010) where it de-aged actor Jeff Bridges, to numerous Marvel and Star Wars movies and streaming shows.
Photorealistic digital re-aging of faces in video is becoming increasingly common in entertainment and advertising. But the predominant 2D painting workflow often requires frame-by-frame manual work that can take days to accomplish, even by skilled artists.
The researchers explain in great detail how they trained U-Net, a convolutional neural network developed by the University of Freiburg for biomedical image segmentation, to analyze and alter subject images. By using upsampling and downsampling techniques combined with many other computations, they used FRAN (Face Re-Aging Network) to create the new images. They have some fairly technical PDFs on the matter that can be found here and here.
“The U-Net architecture of the proposed Face Re-Aging Network (FRAN) takes as input a 5-channel tensor with the RGB image to be re-aged and two additional channels indicating the current and target age of each pixel. Optionally, a pre-trained face segmentation network (BiSeNetV2 [Yu et al. 2021]) can be used to limit re-aging to skin areas and to set localized input and output age values. We use blur-pooling [Zhang 2019] in both FRAN and the discriminator to accommodate small shifts in the positions of wrinkles and other high-frequency details.“
Image Comparisons
“We demonstrate how the simple U-Net, surprisingly, allows us to advance the state of the art for re-aging real faces on video, with unprecedented temporal stability and preservation of facial identity across variable expressions, viewpoints, and lighting conditions. Finally, our new face re-aging network (FRAN) incorporates simple and intuitive mechanisms that provide artists with localized control and creative freedom to direct and fine-tune the re-aging effect, a feature that is largely important in real production pipelines and often overlooked in related research work.“
As Disney demonstrates, FRAN continues to evolve at re-aging actors and it is constantly being refined and trained.
Discussion (8 replies)
Join Discussion →It still looks fake, but at least not as bad as before.
WTH is even Re-Aging? What do you mean re? Why not just aging? You de-age an actor and then re-age them? That doesn't make any sense.
I admit that expression didn't quite sit right with me either. I get what they're trying to say but I feel it should be something different as well.
I was thinking something along the lines of age altering but that can still be confusing because it's the image of the actor being changed, not the actor. It's almost like we need a new adjective/adverb.
Look on the bright side. I'm sure when the knockoff version from overseas comes out we'll all get a good laugh at whatever name they give it.
They talk in the article that it is often done with stand ins anyway, so it is not really aging the actor, but swapping out entirely.
First the pronouns, now the adjectives? please god. :LOL: :p :giggle:
Makes sense that the tech probably can't fill in all the info yet so there's still a human component needed to assist it.