EA Working On Realistically Animating Videos Into Games Using AI

Rundown:

• Electronic Arts is potentially working on a system for generating dynamic animations for video games based on real-life videos using a neural network to extract rough motion data, such as pose information, from a frame (or frames) of a character in motion.
• The system could quickly and efficiently create realistic and fluid animations with minimal input from video game developers.
• The system uses neural networks to access the motion capture video data, receive pose information of the character in motion, and use it to predict the subsequent frame in the animation.
• The patent also mentions that the motion capture video data can be captured from even a mobile device or an actual sports event, which opens up the possibility of creating realistic animations of real-life players in sports games.

Electronic Arts is a multinational video game developer and publisher widely recognised for creating and publishing a wide variety of popular video games across multiple platforms, including sports games, role-playing games, first-person shooters, and more. The company is one of the largest video game companies in the world and has a reputation for developing high-quality and engaging video game franchises that appeal to a wide audience, some of which include Madden NFL, FIFA, Battlefield, The Sims, and Mass Effect. The company is also accredited for its strong partnerships with other companies in the video game industry, as well as its efforts to stay at the forefront of technological advancements in the field of video game development.

In December 2022, we uncovered a recently published patent from Electronic Arts that discusses a system to potentially streamline the creation of realistic virtual worlds in video games by using a neural network to procedurally generate positions of map items, such as buildings and other structures, for placement on a virtual map. Since then, however, a few other patents have been awarded to the company, one of which describes a system for generating dynamic animations for video games based on real-life videos using a neural network to extract rough motion data, such as pose information, from a frame (or frames) of a character in motion.

Earlier today, we came across a recently published patent from Electronic Arts titled “ENHANCED ANIMATION GENERATION BASED ON VIDEO WITH LOCAL PHASE,” which was filed in July 2021 under the name of Electronic Arts Inc. The patent, which was published earlier this month, discusses a system that can potentially generate dynamic animations of characters from real-life motion capture video data, such as pose information. The system, which uses neural networks, could quickly and efficiently create realistic and fluid animations with minimal input from video game developers. According to the patent, the system works by first accessing motion capture video data of a character in motion. The video data is input into a first neural network, which extracts rough motion data for each video frame. This information is then overlaid on the video to generate a modified version. The modified video data is then input into a second neural network, which uses a sliding window of frames to predict the next frame in the animation. The system then moves the sliding window to the next frame, including the predicted frame, and continues to predict the subsequent frame.

“Embodiments of the systems and methods described herein provide a dynamic animation generation system that can apply a real-life video clip with a character in motion to a first neural network to receive rough motion data, such as pose information, for each of the frames of the video clip, and overlay the pose information on top of the video clip to generate a modified video clip,” reads the abstract for the patent. “The system can identify a sliding window that includes a current frame, past frames, and future frames of the modified video clip, and apply the modified video clip to a second neural network to predict a next frame. The dynamic animation generation system can then move the sliding window to the next frame while including the predicted next frame, and apply the new sliding window to the second neural network to predict the following frame to the next frame.”

With this system, the company aspires to streamline the burden on video game developers to create realistic and detailed models of characters and their movements to create immersive gameplay experiences for players. This includes the need to fine-tune character movements, which is time-consuming and complicated due to the hand-tuned approach used to adjust the positions of objects in the character model’s skeleton. However, with the automated system, video game developers may not need to manually fine-tune the character movements, which would not only save resources but also significantly reduce the time it takes for video games to be completely developed, which would satisfy players as well.

According to the claims made by the patent, the motion capture video data that may be input into the system includes a video of a real-life person (or animal) in motion, such as from a sports event, which opens up the possibility of creating realistic animations of actual players in sports games, such as Madden NFL and FIFA. The pose information received from the first neural network includes velocity information corresponding to the joints of the motion capture character, which would then be used to predict future frames from one or more past frames at a predefined time threshold. Interestingly, the motion capture video data doesn’t need to be captured using advanced camera technologies either. “The computer-implemented method of claim 1, wherein the motion capture video is captured from a camera on a user’s mobile device,” states the patent, which means that the system may even be easily accessible to indie developers that don’t necessarily have access to advanced video game development technologies. Additionally, the patent states, “The computer-implemented method of claim 1, wherein the first local motion phase includes phase information for each bone of the motion capture actor,” which means that the system will likely be very detailed as it will supposedly take into account phase information for each bone of the motion capture character.

Electronic Arts has always had a reputation for developing and publishing high-quality and engaging video games that appeal to a wide audience, and this new system is definitely poised to revolutionise how video games are developed, providing players with an even more realistic and immersive gameplay experience while also making it all the more convenient for video game developers to design video games without the need for very advanced technologies. With game engines like Unreal Engine 5 and other potentially forthcoming innovative technologies that companies like Electronic Arts may be working on, the future of video game development seems rather optimistic for companies and indie developers alike. Of course, a patent doesn’t necessarily mean that the method will be implemented, but it’s certainly promising to see companies working towards making video game development more accessible and convenient for everyone.

What do you think about this? Do tell us your opinions in the comments below!

Similar Reads: Sony Bringing Cloud Gaming On Platforms Like Netflix & YouTube