Summary
- Sony is exploring the use of AI to make games more accessible to the visually impaired, adding to the existing efforts in the gaming industry.
- The patent describes an "on-demand accessibility system" that uses modules to audibly describe scenes and provide feedback to the player through the DualSense controller.
- The system may also be applicable to movies, TV shows, and music, and Sony is considering its implementation in future PlayStation games and hardware.
A new Sony patent suggests that AI is being used to make games more accessible to the visually impaired. Video games are becoming more accessible in several ways, and Sony looks to be adding a fresh method of opening the medium up to more gamers.
Sony continues to file new patents to advance its technology across the board. Included among the recent patents are Sony's interest in MMO server tech, a game aroma system, and a unique PlayStation controller. The PS5 and Xbox Series X/S are bridging the gap between gaming and gamers with disabilities, with Xbox's adaptive controller praised by many. Furthering Sony's accessibility, a patent looks at making games that are more digestible by the visually impaired.
According to a new Sony patent, an "on-demand accessibility system" could be implemented in future games to audibly illustrate certain scenes for the visually impaired. This system can be activated by different modules that provide feedback about the game. The modules include an action description module, scene description, color accommodation, graphical style, and acoustic effect annotation. Combining the different feedback that's coupled to the DualSense controller, actions happening on screen can be described to the player through captioning. This new system may provide visually impaired gamers with a more accurate representation of what's happening than a description of each scene in a game.
For a PS5 system to describe things accurately, a neural network, component system encoder-decoder, and memory are used in the process. Sony utilizes data from the DualSense like player movement, changes to frames, button presses, and audio cues played through the built-in speaker. Based on the language of the Sony patent, this new system may be implemented in movies, TV shows, and music as well. Because many considerations have been made by Sony in this patent, open-world games have been looked into. Given the amount of content in open-world titles like Guerrilla Games' Horizon: Forbidden West or Kojima Productions' Death Stranding, the accessibility system may not apply to the genre.
Though some modern games have captioning, this on-demand accessibility system could benefit titles that don't. While this Sony patent may suggest that upcoming PlayStation games could be more accessible, it may take a while before its implemented. Part of the process requires training the neural network to provide the right feedback to players, something that may take a while. With PlayStation looking at features for current and future platforms, hardware in the next console and controller may provide even more opportunities.