Dense Retrievers Know More Than They Can Express (mixedbread.com) AI

Mixedbread argues that dense retrieval models contain richer, “latent” information than their scoring operators can express, and that this hidden vocabulary can be extracted using sparse autoencoders to obtain features that resemble lexical signals useful for retrieval.