Incidental Bilingualism in PaLM's Translation Capability
First page
Paper summary
Explores where PaLM's translation ability actually comes from.
Ask this paper
01
30M+ translation pairs: PaLM is exposed to over 30 million translation pairs across at least 44 languages within its training data, incidentally.
02
Incidental bilingualism: Argues these "accidental" translation pairs substantially explain PaLM's translation capabilities.
03
Scale-of-incidental-data: Highlights how large-scale pretraining can inadvertently cover specialized capabilities via byproducts of web data.
04
Pretraining data insight: An influential study on understanding emergent capabilities via careful data auditing.