Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Preprint
Data sources: ZENODO
addClaim

The Hidden Geometry of Transformer Weights: A Journey Inside the Black Box

Authors: Gernone, Gianluca;

The Hidden Geometry of Transformer Weights: A Journey Inside the Black Box

Abstract

An exploratory journey into a transformer model. We can see how it “breathes,” how it behaves, and where and how to manipulate it. A dedicated knowledge structure that maps the entire model to axes.The canonical-basis realignment is lossless. The realigned model enables 2× geometric compression without calibration data.The black box has a geometry. All it took was looking at it from the right angle.

Powered by OpenAIRE graph
Found an issue? Give us feedback