Researchers led by Lance Ying, including Joshua B. Tenenbaum, introduced AI GameStore, a platform for testing vision-language models on generated adaptations of popular games, in an arXiv paper.
The seven models tested scored less than 10% of the human average on most of the 100 games, according to the paper’s abstract1. The project repository says the human study involved 87 participants and 2,281 sessions, providing the human scores used in the comparison.
The authors say conventional AI benchmarks often test narrow abilities and can saturate as models improve. AI GameStore is designed to add evaluations by using large language models and human reviewers to create standardized, playable versions of games sourced from Apple App Store and Steam charts. The 100 games are generated adaptations, not the original commercial games, so results on them do not establish how models perform in those originals.
The abstract says the strongest models struggled particularly with tasks involving world-model learning, memory and planning. It proposes comparing AI and human players with similar experience, time or other resources, but the abstract does not establish whether the reported tests matched players on those conditions.
The full paper PDF was unavailable for review, so the reported aggregate score could not be checked against individual-game results. The findings here come from the abstract and the project repository2. The arXiv page’s author-supplied comments field says the paper was published at NeurIPS 2026; that publication claim was not independently verified.
The repository lists 10 games as public and the remaining 90 as a private test set intended to limit benchmark contamination. It also asks users not to train AI models on the games, gameplay videos or study data, and says that request is not a license condition. The repository says the complete suite can be used to evaluate a model through the AI GameStore website.