Is Cosine-Similarity Really About Similarity?
This paper argues that cosine similarity between learned embeddings does not always measure semantic similarity, and gives analytical examples where it produces arbitrary or non-unique values.
Ask this paper
Analytical setting: Studies embeddings derived from regularized linear models, where closed-form expressions expose how cosine similarity depends on regularization choices.
Arbitrary "similarities": Different but equally valid optimal solutions can yield wildly different cosine similarities between the same input pairs, undermining the metric's interpretability.
Implicit regularization effects: Common deep-learning regularizers induce unintended effects on cosine similarity that practitioners often don't notice or control for.
Recommendations: The authors urge caution in using cosine similarity as a universal semantic metric and suggest alternatives such as task-calibrated similarity functions or metric learning.