Skip to content
agncrowPublic

About

PyTorch written in Crow: networks, optimisers, data, statistics and models, every function held to PyTorch's numbers in float64 and every answer the same interpreted and compiled.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Pie

Crow's numbers beyond the language's own, written in Crow: networks and their training, as PyTorch builds and trains them — tensors' layers and losses, a model as one parameter vector differentiated by grad, the optimisers and their schedules, datasets and batches, saved weights — with the statistics, the classical models, the optimisers of scipy, symbolic regression, graphs and retrieval beside them. Every answer is the same in crow run -i and crow run, and each package is held to the library it follows: PyTorch in float64, scikit-learn, scipy, networkx, faiss.

Alpha. Pie is open source under the MIT or Apache-2.0 licence, at your option (LICENSE-MIT, LICENSE-APACHE). It needs crow 0.9.0 or later, the first release with the fixes Pie's packages need. Pie takes no pull requests during the alpha, and bug reports are welcome in Issues (CONTRIBUTING).

Package What it is
packages/nn activations, losses, layers (linear, convolutions, poolings, paddings, resamplings, RNN/LSTM/GRU, attention, transformers, norms, embeddings, dropouts), modules, initialisers, optimisers and their schedules — PyTorch's torch.nn, torch.nn.functional, torch.nn.init and torch.optim
packages/stats descriptive statistics, correlation, tests and fits — scipy.stats
packages/models the classical models, metrics, cross-validation — scikit-learn
packages/optimize minimisers and roots — scipy.optimize
packages/symbolic symbolic regression — PySR
packages/gpt nanoGPT's model and training on nn, its checkpoints by their names — held to nanoGPT
packages/graph graphs, paths, ranks — networkx
packages/retrieval TF-IDF and BM25 — scikit-learn, rank_bm25
packages/vectors an exact vector index — faiss

A program installs a package as any dependency and calls it qualified:

deps = map[nn = 'github.com/agncrow/pie/packages/nn@v0.1.0']
use math
use nn
use nn.functional as F
use nn.optim, nn.rng

fn loss[model, theta, x, y]
    return F.mse_loss[nn.forward[model, theta, x], y]

model = nn.sequential[[nn.linear[3, 4], nn.relu[], nn.linear[4, 1]]]
x = tensor[[[1.0, 2.0, 3.0], [0.5, -1.0, 2.0]]]
y = tensor[[[1.0], [0.0]]]
opt = optim.sgd[nn.parameters[model, rng.manual_seed[0]].values, lr = 0.1]
for k in range[50]
    opt = optim.step[opt, grad[loss][model, opt.theta, x, y][0]]
log[loss[model, opt.theta, x, y]]

A model is a record of its layers beside one vector of parameters, theta, drawn from a torch generator (nn.rng) as PyTorch's modules draw it — the same numbers, after torch.manual_seed(0) in float64; grad[loss] answers the gradient of every parameter at once, and an optimiser's step is arithmetic over that vector. The same model and steps in PyTorch, in float64 from torch.manual_seed(0), end at the loss this prints to twelve digits. A loader (nn.data) hands out PyTorch's batches in PyTorch's order, and nn.safetensors saves and loads weights in the format PyTorch models are shared in.

A program that writes gpu[] before it makes its model trains it on the GPU with nothing else changed, every tensor and every gradient staying there; the GPU computes in f32, so its answers agree with the CPU's to f32's digits. A perceptron 1024 wide on a batch of 512 trains about ten times faster there; a small model trains faster on the CPU (docs/pie-design.md, On the GPU).

./test.sh [crow] runs every package's tests and examples in both modes. docs/pie-design.md says where the line with crow lies and why a model is one parameter vector.

Licence

Pie is licensed under either of the MIT licence (LICENSE-MIT) or the Apache licence, version 2.0 (LICENSE-APACHE), at your option, as Crow is. Each package carries both files, since crow install copies a package's folder alone.

About

PyTorch written in Crow: networks, optimisers, data, statistics and models, every function held to PyTorch's numbers in float64 and every answer the same interpreted and compiled.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages