Figure duality · A Small Switch in a Vast Machine

Figure viewer

Back to article ↩

← Back to article

A weight-space step maps through J to a blue activation-space step. A shrinking Fisher ellipse centered on the desired change first touches the reachable line at that same endpoint. — Metric projection · Exact mathematical model
With a complete interface and its pulled-back Fisher metric, the two routes agree locally: move the weights along their natural gradient, or project the activation natural gradient onto reachable changes.
Settings, scope and source files
Exact tangent model: $F=\begin{pmatrix}2&.5\\.5&1\end{pmatrix}$ , $v=(.8,1.4)$ , and permitted changes $u_2=0$ . Contours show $\tfrac12(u-v)^\top F(u-v)$ ; first contact with the blue line gives the Fisher projection $(1.15,0)$ , distinct from the Euclidean projection $(.8,0)$ . With $J=(1,0)^\top$ , the weight metric is $G_\theta=J^\top FJ=2$ . For the linear objective with differential Fv, the weight natural gradient pushed through J equals that projection. The two arrows grow together to display this identity. This is an analytic construction, not Qwen measurements or an optimization record.
Source asset (original) ↧ Source asset (GIF) ↧ Source asset (MP4) ↧