LLaVA-RLHF
Free while signed in. Answers cite the passages they came from.
Adapts factually augmented RLHF to aligning large multimodal models, reducing hallucination without falling into reward-hacking pitfalls.
Factually augmented RLHF: Augments the reward model with factual-consistency signals (e.g., grounded-in-image checks), reducing the reward hacking common in vanilla multimodal RLHF.
Hallucination reduction: Produces meaningful reductions in hallucination on multimodal benchmarks compared to SFT-only or vanilla RLHF variants.
94% of text GPT-4: Reaches 94% of the performance level of text-only GPT-4 on LLaVA-Bench - closing a substantial gap via alignment alone.
Open recipe: Releases the full training recipe so the multimodal RLHF approach can be applied to other open VLMs.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack