Training

Baize: An Open-Source Chat Model with Self-Chat Data

First page

Paper summary

An open chat model fine-tuned with LoRA on self-chat dialogs.

Ask this paper

Key points

Self-chat data generation: Generates 100K dialogs by having ChatGPT converse with itself, then fine-tunes on these dialogs.

LoRA fine-tuning: Uses parameter-efficient LoRA fine-tuning for compute efficiency.

Multiple model sizes: Releases 7B, 13B, and 30B parameter models along with the dialog data.

Open chatbot ecosystem: Part of the 2023 proliferation of open chat models (Vicuna, Alpaca, Koala, Baize) building on LLaMA.

Every Monday