Skip to content

Latest commit

 

History

48 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Gradient Descent in Weight Space Is Kernel Descent in Activity Space

A short paper showing that gradient descent on network weights induces kernel descent in the space of neural activities, governed by a neural-tangent-kernel-style Gram matrix on internal neurons.

Key result: When the kernel is diagonally dominant (wide networks), each neuron's activity change is approximately proportional to the negative loss gradient with respect to that neuron's activity — converting untestable claims about synaptic learning rules into testable predictions about observable activity changes.

Reproduce the results

Install uv, then run the complete dependency graph from the repository root:

uv run snakemake --cores 1

This regenerates seeded simulation outputs, publication-ready figures, manuscript assets, and the compiled paper. The Snakemake dependency graph is the authoritative provenance record. To rebuild one step, use uv run snakemake --cores 1 --forcerun <rule_name>.

Repository layout

Interactive Demo

Launch the interactive demo — the runnable demo is the static browser page in index.html, served directly by GitHub Pages with no install or build step. Adjust width, depth, and learning rate to see how the kernel and diagonal approximation behave.

About

Gradient descent in weight space is kernel descent in activity space

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages