Safety

Concept Scrubbing in LLM (LEACE)

Free while signed in. Answers cite the passages they came from.

First page

The curator’s take

Least-squares Concept Erasure - erases a target concept from every layer of a neural network.

Key points

Closed-form erasure: Provides a closed-form solution for removing linearly-encoded concepts (like gender) from representations at every layer.

Theoretical guarantees: Mathematically guarantees the concept cannot be linearly recovered after erasure.

Bias reduction: Applied to reduce gender bias in BERT embeddings while minimizing impact on other capabilities.

Interpretability tool: Became a standard tool in the model-editing and interpretability literature for studying what information models use.

Every Monday

The same picks and the same summaries, in your inbox. Free, and 176 issues deep.