SIMA
DeepMind's Scalable Instructable Multiworld Agent (SIMA) is a generalist AI agent that follows natural-language instructions across nine commercial 3D video games like No Man's Sky, Teardown, Valheim, and Space Engineers.
Ask this paper
600-skill taxonomy: Evaluation covers 600 basic skills spanning navigation, object interaction, and menu use, giving a detailed picture of what generalist 3D agents can and can't do.
Training data: Built by recording human players instructing each other in gameplay and fine-tuning a pretrained vision-language backbone on this instruction-paired footage.
Cross-game generalization: Training across many games produces an agent that performs nearly as well on unseen games as on games it trained on - a strong generalization signal for 3D embodied agents.
Language matters: Ablations show that textual instruction is the dominant performance driver; removing the language conditioning collapses skill execution, underscoring language as an alignment interface for embodied agents.