Compositional Foundation Models (HiP)
Free while signed in. Answers cite the passages they came from.
Proposes foundation models that compose multiple expert foundation models trained on different modalities to solve long-horizon goals.
Hierarchical planning: Uses separate foundation models for language (high-level plans), vision (grounding), and action (execution) that compose into a hierarchical planner.
Long-horizon goals: Targets goals requiring dozens of subgoals - a regime where monolithic policies typically fail.
Training-free composition: Composes existing pretrained models at inference time without joint training, dramatically reducing the compute cost of long-horizon agents.
Robotics relevance: Demonstrates the approach on robotic manipulation tasks, pointing toward practical long-horizon embodied-AI systems.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack