Gaming as the Training Ground for Physical AI

For years, gaming has served as a sandbox for AI experimentation. Now, a pivot is occurring: the industry is becoming the primary gym for training the robotics of the future.

As reported by Wired, researchers including Yann LeCun and Fei-Fei Li are focusing on "world models." Unlike large language models (LLMs) trained on text, world models require visual and action data to navigate the physical world. However, a lack of cause-and-effect data on the internet has created a bottleneck. Xiatian Zhu, an associate professor at the University of Surrey, notes that this specific type of data is scarce online.

To solve this, British startup Worldmodeldata—advised by LeCun—is brokering controller inputs and data from game studios to create training datasets. CEO Rhea Loucas tells Wired that the company has licensed nearly 1 million hours of data from unnamed studios, wagering that the diversity of gaming experiences can teach AI to handle the "corner cases" of the real world. This sentiment is echoed by Nicole Fraenkel, a partner at Khosla Ventures, who notes that manual data collection cannot capture the disorder of the real world, whereas gaming data is available in massive quantities.

However, the transition from screen to street isn't without friction. Ming-Yu Liu, who leads world model development at Nvidia, argues that game physics are often "eccentric" and lack the fine-grained detail required for complex motor control, such as gripping an object. While companies like Niantic and General Intuition are already leveraging their own platform data, the industry remains divided on whether gaming data is best suited for robotic manipulation or the generation of hyperrealistic 3D environments.