Sony has filed a patent that aims to turn bananas and other random objects into gaming controllers. The engineering challenge was spotted by GamesIndustry.biz, which shared excerpts from a recently published application explaining how a camera could be used to track and map virtual buttons onto passive items to transform them into steering wheels, flight sticks, and other kinds of control devices. It isn’t clear why Sony chose to use bananas to demonstrate the technology specifically, but if the idea pans out, PlayStation gamers could find themselves using a pair of fruit to control Kratos’ Blades of Chaos one day, among other creative uses.
“A camera gets images of the items in users’ hands, and tracks the items based on pixels, contours, and/or colors in the images rather than QR codes or other such tracking techniques,” GamesIndustry.biz explained. “A game could either be trained to recognize objects as controllers, or pre-configured such that users are told ahead of time what could be used as a controller. The application doesn’t mention it specifically, but this latter application sounds like a more cost-effective way to approximate toys-to-life functionality in games.”
“The application goes on to discuss ways to infer the banana’s movement in 3D space, which might be used to control the in-game camera, replace a flight stick, or pause the game if the banana were to be set down out of the camera’s view. […] this patent application [also] proposes mapping virtual buttons onto the banana, possibly with the help of a camera mounted in a virtual reality headset.”
GamesIndustry.biz also shared another patent application for what could very well be Sony’s next-generation PlayStation VR controllers. Titled “control apparatus and control program,” the patent includes diagrams for two hand-held peripherals that feature small LED panels in the place of the original controller’s light-up globe. There’s also a new trigger button that seems to leverage the same haptic feedback technology in the DualSense wireless controller.
