Google DeepMind on Tuesday released its latest global model, Genie 4, as a research preview. The latest model produces the most photorealistic and interactive world ever, allowing users to change the scene with prompts during the simulation. The model also generates real-time audio that matches the simulated world, including music, ambient noise, and interactive dialogue.
The release of Genie 4 comes a few months after AI video startup Runway released its second world model, GWM Worlds 2. GWM Worlds 2 is a playable model that dynamically responds to user input and includes audio generation. At the same time, the global model industry is beginning to consolidate. Fei-Fei Li’s World Labs was acquired by AMD, Amazon is in talks to acquire Decart, and Rivian just acquired Third Dimension to bring simulation capabilities in-house for self-driving testing.
Genie 4 is Google’s latest effort to gain an edge in a crowded field.
Google released its predecessor, Genie 3, as a research preview a little over a year ago. Genie 4 is much more photorealistic and has audio, as well as free-form editing, including changing weather and camera angles in real time, and adding new people and objects. It can maintain a consistent scene for up to two minutes, which is double the previous version’s limit, and offers better physics and more human-like characters.
Gemini, Google’s AI model, is also more tightly integrated with Genie 4 and acts as a director, planning steps and checking frames while Genie creates visuals. In research experiments associated with Gemini Robotics, Gemini 4’s argon model broke down tasks into steps, which Genie 4 simulated in real time. This experiment demonstrates how robotic scenarios could potentially be tested in simulation before they occur in the real world.
“What we’re going to see more and more of is a hybrid world model that has a combination of Gemini writing code and text, Genie ingesting it in real time, and simulating reality at different levels of abstraction,” Tejas Kulkarni, Google’s research director, said in a press conference.
Similar to other world models, proposed commercial use cases include gaming, interactive learning, robotics, etc. One speculative example of interactive learning is to enter a painting depicting a historical scene and have a conversation with the painting’s subject about the depicted event.
But “trusted testers” — the people who have first access to Google’s cutting-edge products — point to a potential new use for film prototyping.
“The real-time aspect is most valuable in areas where maintaining a flow state in the creative space is extremely helpful,” Google DeepMind engineer Diego Rivas said at a press conference. “For example, whenever you need to generate ideas quickly, such as during the ideation stage or storyboarding stage.”
Google also offers Veo, an AI video generation model that creates high-resolution clips with synced audio from text, images, or video prompts. This tool is becoming increasingly popular among filmmakers.
Rivas added that users will eventually be able to turn those prototypes into creative assets with Gemini Omni, a multimodal large-scale language model that can generate video from images, audio, and text. However, he says these capabilities are still a long way off.
Throughout the briefing, Rivas emphasized that Genie 4 is still in its early stages and it is not yet clear what the end-to-end product experience will be. The team also acknowledges that there are still gaps in Genie 4. As the simulation runs longer, details and objects can shift, and we still don’t fully understand how to ensure that audio is accurate and text in the world is readable.
Genie 4 is currently available as a research preview for trusted testers. In the coming weeks, this model will be rolled out to Google AI Ultra subscribers in Project Genie, Google’s AI tool that generates interactive worlds from text and image prompts, and early access partners in the Gemini API and Google AI Studio.
If you make a purchase through links in our articles, we may earn a small commission. This does not affect editorial independence.
