Back Propagation Neural Network

Full usage, build, TUI, format and troubleshooting reference: User Manual — manual/USER_MANUAL.md

• The goal of any supervised learning algorithm is to find a function that best maps a set of inputs to their correct output. The motivation for backpropagation is to train a multi-layered neural network such that it can learn the appropriate internal representations to allow it to learn any arbitrary mapping of input to output. See Wikipedia and Neural Network Architectures & Deep Learning.

• Neuron/perceptron: the basic unit of the neural network. Accepts an input and generates a prediction. Neural networks generate their predictions in the form of a set of real values or boolean decisions. Each output value is generated by one of the neurons in the output layer.

• Activation function example: non-linear Hyperbolic Tangent, zero centered making it easier to model inputs that have strongly negative, neutral, and strongly positive values. See video Types of activation functions.

Simulation

Training sets live as plain-text files in the data sub directory, grouped into families of related tasks:

• Boolean gates: XOR, OR, AND, NAND, AND3.
• Grid pattern classification: 2×2, 3×3, 4×4 and 5×5 pixel grids that classify solid / vertical / diagonal / horizontal / X / cross patterns (the larger grids also count solid pixels and detect noise).
• Bit counting: count the number of high bits in the (scaled) input.

Every set uses the same script format shown below, so a file can be swapped in without touching code.

Refactoring to Modern C++

Source code: David Miller, C++ code example, also available in src-original directory. Associated video: David Miller, Neural Net in C++ Tutorial Goal of this project: refactoring the David Miller example code to Modern C++. Actively maintained; more improvements are planned.

The refactoring has already applied a number of modern C++ techniques:

• C++20 as the language standard (-std=c++20).
• std::size_t types for indices and sizes of containers (no more unsigned mix-ups).
• Structured bindings, digit separators (1'000'000) and brace initialization.
• inline constexpr constants for training defaults, centralized in src/NNconfig.h.
• [[nodiscard]] on value-returning accessors to catch discarded results.
• RAII via OSstate to restore stream formatting flags automatically.
• std::format for error messages.
• Errors reported through std::runtime_error exceptions instead of exit().
• Backpropagation correctly skips the bias neurons: the output layer's bias has no target value and the next layer's bias has no incoming weights, so backProp() and sumDOW() iterate only the real neurons.
• Weight initialisation uses a seeded Mersenne Twister (std::mt19937) with Glorot/Xavier init for tanh/sigmoid layers and He init for the ReLU family, replacing the non-centred uniform [0, 1) default. A fixed seed makes training runs reproducible.

Compiling

The C++ code needs c++20 and cmake (version 3.20 or newer) to be installed.

Go to the build directory and type:

cmake ..
make -j

FTXUI (v5.0.0) is pulled automatically via FetchContent and patched for a canvas negative-size crash (patch in cmake/); the dashboard also links ftxui::component.

The executable backpropnn is written to the bin directory.

Alternatively build with the standalone Makefile (compiles with an extra -Weffc++):

make -C src

Use cppcheck

cppcheck --enable=all --std=c++20 --verbose .

Running and training a BackProp NN

A set of inputs for which the correct outputs are known, used to train the neural network.

Training XOR, topology:

• 2 inputs
• 1 hidden layer 5 neurons
• 1 output neuron

Empty lines and single line comments after # are allowed.

The training parameters momentum and learning_rate are optional. If not used, the default values 0.5 and 0.15 (defined in src/NNconfig.h) are applied. Training stops when the recent average error falls below 0.03, or after 1,000,000 training passes, whichever comes first (both limits are defined in src/NNconfig.h).

The weights are initialised with a seeded Mersenne Twister using Glorot/Xavier init for tanh/sigmoid layers and He init for the ReLU family. The optional seed parameter controls the random generator (default 1, see src/NNconfig.h): the same seed and config produce an identical, reproducible training run.

The labels momentum and learning_rate may also be written as ALPHA and ETA.

For every layer (except inputs) the activation function can be selected:

• tanh
• sigmoid
• relu
• leaky_relu

Training script example:

# trainingXOR.txt

momentum: 0.5
learning_rate: 0.15
seed: 1

topology: 2      5    1
actionfs: inputs tanh tanh

in: 0.0 0.0
out: 0.0

in: 1.0 0.0
out: 1.0

in: 0.0 1.0
out: 1.0

in: 1.0 1.0
out: 0.0

show_max_inputs: 2
show_max_outputs: 1
output_names: XOR

Errors in the training script (unknown labels, out-of-range values, a topology that does not match the activation function list, ...) are reported with the offending line number and stop the program with a non-zero exit code. Example:

ERROR: === ERROR line [5]: 'bogus_label:' ???

Go to the bin directory and run the code for training XOR:

./backpropnn ../data/trainingXOR.txt

At the end of the training the next text will be shown:

- Results after training:     

+0.000 +0.000 
===> 
XOR  +0.003 

+1.000 +0.000 
===> 
XOR  +0.984 

+0.000 +1.000 
===> 
XOR  +0.983 

+1.000 +1.000 
===> 
XOR  +0.020 

Live training dashboard (TUI)

When run on a real terminal, the executable renders a live interactive dashboard powered by FTXUI, driven by an FTXUI event loop on the main thread while the trainer runs on a background thread. The dashboard shows the network settings, iteration progress with a passes-per-second speed and ETA, average/best error gauges, a semi-log error-history plot (with best/now markers), the current input sample as a 2-D viridis colour heat grid, per-sample output magnitude bars, a live accuracy score, a weight heat map of the L0→L1 connections (on wide terminals), and a RUNNING/PAUSED/DONE status badge:

# on a real terminal (auto-detected), ideally ≥ 40 rows
./backpropnn ../data/trainingXOR.txt

The layout is responsive: the plot and settings panels resize with the terminal, the Features L0-L1 weight heat map appears when the terminal is wider than 130 columns, and the whole dashboard scrolls when the content is taller than the terminal (or via the mouse wheel). Press h/? for an in-terminal help overlay listing every key.

The classic plain-text output (useful for scripts and logs) is automatically used when stdout is piped or redirected:

./backpropnn ../data/trainingXOR.txt > results.txt   # classic output

Interactive keys

Key	Action
q, Esc	Stop training (if still running), leave the dashboard, print a one-line summary
p, Space	Pause / resume training (shown in the status badge)
h, ?	Toggle the in-terminal help overlay
↑/↓, j/k, PgUp/PgDn, Home/End, mouse wheel	Scroll the dashboard when its content overflows the terminal
+, =	Render the dashboard half as often (fewer passes between redraws)
-, _	Render twice as often (more passes between redraws)

When training ends on its own, the dashboard stays on screen showing the DONE badge until you press q/Esc. On exit (naturally or with q) a summary is printed, e.g.:

- Training done: 1802 passes, avg error 0.030075, best error 0.030075, 13.5 ms, accuracy 97.6% (1759/1802)

q/Esc prints the same line with an (interrupted) marker and the elapsed wall-clock time.

Runtime control

The dashboard can be forced on or off at run time with the BPNN_TUI environment variable:

Value	Behaviour
BPNN_TUI=0	Always use classic plain-text output
BPNN_TUI=1	Always render the TUI (even when piped)
(unset)	Auto: TUI on a terminal, classic when piped

Build without FTXUI

FTXUI is pulled automatically via FetchContent during a CMake build. To build without the dependency and the dashboard:

cmake -DBPNN_TUI=OFF ..
make -j

The standalone Makefile (make -C src) also compiles cleanly without FTXUI; the TUI code compiles to a no-op.