Waypoint-1 Overworld

Waypoint-1 is a real-time interactive video diffusion model developed by Overworld, designed to generate dynamic and interactive virtual worlds. It allows users to step into and interact with a created world by providing initial frames and then controlling the model via text prompts, mouse movements, and keyboard inputs. The model's core is a frame-causal rectified flow transformer, trained on an extensive dataset of 10,000 hours of diverse video game footage, coupled with corresponding control inputs and text captions. Unlike many existing world models that fine-tune pre-trained video models with simplified controls, Waypoint-1 is specifically trained from the ground up for interactive experiences. This approach enables it to offer zero-latency control, allowing users to freely move the camera with a mouse and input any key on the keyboard, with each frame generated in context of these real-time controls. The model is also optimized to run efficiently, providing a seamless experience even on consumer-grade hardware. The inference backbone for Waypoint-1 is WorldEngine, Overworld’s high-performance library built for interactive world model streaming. WorldEngine is designed for low latency and high throughput, providing core tooling in pure Python for building inference applications. It processes context frames, keyboard/mouse inputs, and text to output image frames for real-time streaming, achieving impressive frame rates even with complex operations.