It’s National Video Game Day! To celebrate, we’re exploring innovation in this exciting area by taking a look at patent trends in video game technologies at the European Patent Office (EPO).
Future trends
A recent report by the Interactive Software Federation of Europe (ISFE) puts the 2018 market size for the video game industry in Europe at €21bn, with a year-on-year growth of 15%. It’s therefore no wonder that the biggest players in the industry want to protect the next generation of hand-held controllers, software and consoles. By looking at publication and grant data for applications at the EPO over the last 10 years, we can observe the technological trends in this time and try to predict what the future might have in store for gamers.
Three key areas are clearly driving the next generation of gaming tech, and these areas promise to deliver an entirely new gaming experience in the near future.
Snapshot: video game patents at the EPO
Virtual Reality controllers – more than just fingers and thumbs
The first patent trend: using sensors to control a game in a virtual reality environment.
VR has fronted a new style of gaming, and we’ve come a long way since the Nintendo ‘Virtual boy’ in 1995. There are now plenty of VR headsets on the market that allow users to control an in-game field of view by moving their head, meaning players can become more immersed in a game than ever before.
As an example, an application filed by Samsung describes a sensor worn by a user to obtain biometric data, which is then used to generate a more realistic virtual world. A sensor might measure a player’s level of brain activity, which is then applied in game to represent an in-game ‘concentration’ level. The application doesn’t specify which games could be enhanced using a player’s concentration level, although our vote goes to a VR version of ‘Operation’! Other possibilities include breathing sensors being used to steady a weapon in a first-person shooter (much more realistic than holding down a button to hold your breath in-game!).
Wearable sensors - numbers of applications published per year
The number of applications published by the EPO per year classified under ‘Input arrangements for game devices, using sensors worn by the player’ (CPC classification A63F13/212)
Another application filed by Samsung relates to applying sensors to a VR headset that can visually recognize specific hand gestures of the user and selectively ignore the hand gestures of an ‘intruder’ (the person not wearing the headset), such that only the player can affect the game with hand movements. Another application describes a ‘bending sensor’ worn on the elbow of a player that is used to control an avatar in a virtual world, by sensing a bending of the player’s arm and mapping this to movement of the avatar’s arm.
Sony even anticipate that a player may become too immersed in a VR game. A recently published application addresses the problem of player’s dropping their game controller during gameplay, mistakenly thinking they are handling a virtual in-game object. Sony describe multiple solutions, including using pressure sensitive sensors that can sense an ‘imminent dropping’ of the controller by the player, allowing the system to warn the player and bring them back to reality. Perhaps this technology might one day be used to stop us rage-throwing our controllers at the screen, too…
Immersive feedback – it goes both ways
The second patent trend: new ways of providing physical feedback to the player.
Haptic, or tactile, feedback has been around for a while and will be nothing new to the seasoned gamer. Nintendo’s ‘Rumble Pack’ (April 1997) and Sony’s DualShock controller (August 1997) were arguably the first mainstream products to introduce a vibration feedback effect to the home gaming scene. With the technological and commercial success of these systems came patents (Nintendo’s ‘Rumble Pack’ application makes for interesting reading) and patent litigation (in the form of the famous Immersion v. Sony lawsuit).
Haptic feedback - numbers of applications published per year
The number of applications published by the EPO per year classified under ‘generating tactile feedback signals via the game input device’ (CPC classification A63F13/285)
Looking at recent publications classified under the heading ‘generating tactile feedback signals via the game input device’, it appears that we are now heading towards a new age of tactile feedback, which can provide a much more complex and personal feel to a game. A recently published application by Sony appears to be building on the DualShock technology by providing an ‘external vibrating unit’ that can be attached to a game controller. The attachment can deliver additional feedback functionality, such as alternative modes of vibration response to different feedback signals from the game. In one example, a very recently published Immersion Corp application describes a device worn on the finger and thumb of a player that provides in-game haptic feedback to the player. The device simulates a force to the players hand applied in response to the player grasping a virtual object in the game, making it feel like the player is grasping a real world object.
Moving away from the traditional feedback applied through handheld controllers, another Immersion Corp application published last year relates to other types of wearable articles that provide haptic feedback to a user, where the ‘wearable article’ can range from anything from a pair of gloves to a whole body suit. Implementations of ‘haptic actuators’ used in the wearable articles include heating elements, pneumatic and hydraulic actuators and even ‘electroactive’ materials that change shape when subjected to an electric field. Indeed, there are already products such as the full body suit produced by Teslasuit that provide a full body experience, and now much of the focus appears to be on making haptic feedback as realistic as possible.
AI in games - smarter than you think
The third patent trend: using AI and machine learning (ML) to dynamically improve gameplay.
We couldn’t prepare an article on current and future technology in video games without mentioning the ever-increasing importance of AI’s role in improving gameplay.
Adaptive learning - numbers of applications published per year
The number of applications published by the EPO per year classified under ‘Generating or modifying game content before or while executing the game program adaptively or by learning from player actions’ (CPC classification A63F13/67)
Whilst the number of published applications in this area is fewer than in the areas mentioned above, the average number of applications has increased significantly in the last decade – a trend we think will continue into the next decade and beyond.
There is a misconception that the EPO is not friendly toward either AI or video game patents, which may explain the lower figures. However, as any European patent specialist in this area will tell you, it is certainly possible to obtain European patents in this area, though applications must be drafted and framed in a particular way to help maximise the chances of obtaining grant in Europe. This discussion is outside the scope of this article, but the EPO have recently published guidance on its approach to assessing whether an AI-related invention may be considered technical – you may read that guidance here: EPO Guidelines G-II 3.3.1 and our comments on what this might mean for AI inventions in Europe here: Assessing the EPOs new guidelines on AI.
Many of the applications we found relate to the use of AI to improve the gameplay in new ways. For example, an application filed by Sony describes a gameplay assistant powered by AI that learns the available paths taken by previous players, learns how different players play the game, and then uses its new-found knowledge to help players navigate their way through the game. We imagine this assistant to be like a high-powered ‘Navi’ the fairy, though hopefully less annoying. In another example, this recently published application filed by Nokia describes a way of using a neural network to generate high resolution VR content in regions of the environment outside the user’s viewport, which is a solution to the difficulties in streaming high-resolution VR environments.
The benefits of AI are not limited to gameplay itself. For example, Sony have identified the issue with ‘toxic’ players whose behaviour can negatively impact other players’ online gaming experience, for example through aggressive or offensive interactions. This recently published application describes a way of using machine learning techniques to identify potential ‘toxic’ players by characterising and identifying patterns in their behaviour. The application suggests that toxic players could face bans from online services, removing the need to manually report players, whilst players with ‘positive’ behaviour can also be automatically identified using the same method and rewarded with in-game items.
Loading the next level… what next for games?
It’s no surprise that VR and AI are driving innovation in video games, but some of the new ideas and the way they could impact future games are fascinating. The AI-generated elements of a new game might include dynamic game-difficulty balancing, more realistic player-vs-NPC combat, and even dynamic content generation tailored to a specific player’s abilities and interests. Just as AI and ML techniques can be used to write Harry Potter Chapters and compose music, advances in AI and ML in the gaming world mean that AI is beginning to seem less a useful tool for game designers, and more part of the design team in its own right.
The explosion of virtual reality onto the scene in recent years is also driving a large proportion of future gaming developments, as companies look to provide an increasingly realistic and immersive experience. It seems that the virtual reality world Oasis in Steven Spielberg’s Sci-Fi motion picture Ready Player One, featuring an entire open VR world and full body haptic feedback suits, may come to life much sooner than we had thought!
For more information or advice, please contact Tom Hamer, Matt Woodhill or your usual Kilburn & Strode advisor.