Francisco Ferreira da Silva

Compressed Computation under $L^4$ Loss is likely Computation in Superposition

Francisco Ferreira da Silva, Stefan Heimersheim

Neural networks are thought to represent concepts as directions in their activation space, and superposition lets them encode more concepts than they have dimensions. It is natural to ask whether they can also compute more functions than they have neurons, i.e., perform computati…Read more