The benchmark improvement is the original authors’ result. Our E1 instead checks numerical agreement between a weight edit and an activation hook on 40 fixed prefixes; it does not repeat the full benchmark.
Settings, scope and source files
Left: the original authors’ reported MATH-500 accuracy. Right: project E1, comparing a weight edit with an additive activation hook on 40 fixed prefixes. Logit units and KL (in nats) use separate logarithmic axes. This is not a new reproduction of the full benchmark.