Microsoft Research, in collaboration with Ninja Theory (known for the Hellblade trilogy), created Muse, the first generative AI model that can create game environments based on visuals or player actions. It was trained on the failed online game Bleeding Edge.
Microsoft Research, in collaboration with Ninja Theory (known for the Hellblade trilogy), created Muse, the first generative AI model that can create game environments based on visuals or player actions. It was trained on the failed online game Bleeding Edge.
Microsoft’s blog post describes this AI as the first World and Human Action Model (WHAM). It notes that Muse could help Xbox developers create parts of games in the future — the company says it’s even exploring the potential of using it to preserve classic games and optimize them for modern gaming hardware.
Muse AI «understands» the three-dimensional game world and game physics and can respond to how players interact with the game.
«This allows the model to create consistent and varied gameplay, demonstrating a significant step towards generative AI models that can empower game creators,» explained Fatima Kardar, corporate vice president of gaming AI at Microsoft.
The Muse model was trained on a large amount of human gameplay data from the Xbox game Bleeding Edge, thanks to a collaboration between Microsoft Research and Xbox studio Ninja Theory — both based in Cambridge, UK. The AI was trained on data equivalent to seven years of human gameplay, resulting in a billion action-image pairs in total.
The project’s results were published in the journal Nature, but because it’s early work, the model is limited to generating gameplay visuals at a resolution of just 300×180 pixels. That’s higher than the 128×128 resolution of Microsoft’s previous work on generative AI game models, but still well below the 1080p (1920×1080) resolution common among PC gamers.
Microsoft has released several examples of the Muse model in action, generating gameplay and even allowing players to upload visuals into the game to prompt the models. This could be used in early game iterations, but Microsoft emphasizes that it’s not intended to create an entire game and replace developers.
Microsoft also showed off a live version of Muse. The demo generated game-like visual effects on the fly in real time, and even reacted to objects thrown into the game to change the environment. While the real-time gameplay only ran at 10 frames per second and a resolution of 300×180, it was an early demonstration of what might be possible in the future.
Recall that last fall, billionaire Elon Musk announced that corporations own many game studios, and he would create his own with the help of his AI company xAI.
«xAI is going to launch an AI game studio to make games great again!» Musk declared. He recently reaffirmed that he is starting an AI game studio.
In 2023, the largest digital game store, Steam, refused to allow developers to host projects that used AI-generated content. At the time, Valve, which owns Steam, said it needed «time to learn» the changes in the technology. It later announced that it had developed an approach to such games and made changes to its rules that allow the publication of most of them.
dev.ua asked Ukrainian game developers what they think about this innovation and about games with AI content in general. Here’s what they told us.
