Documentation

Versions

Every release keeps its own copy of the documentation, so a link to a particular version does not change meaning when a later one is published. This page is the index, and the record of what changed between them.

#Releases

Verified on this machine with Apple clang: v0.0.1 under -std=c++17 -Wall -Wextra, v0.0.2 and v0.0.3 under -std=c++20.

#Which One To Use

Training a real model in C++ → v0.0.1

It is the only release with mini-batching and dropout, L2, a validation split, seeded reproducibility, multi-class softmax and model save/load — and the only one whose test suite passes. Its limitation is speed, not capability.

Calling it from Python → v0.0.2

The only release whose pybind11 module builds. Per-sample SGD only, and none of v0.0.1’s training features.

Working on the engine → v0.0.3

The current line of development: flat memory, float, SIMD pragmas, batching. It compiles and trains, but has no working tests, so treat it as unverified.

#What Changed, Release by Release

Every row is a property of the engine; every cell is what that release does with it. Cells in bold mark where a release changed the previous answer.

#Architecture and memory

#The hot path

#Training

#Model capability

#Safety and verification

#Defects fixed, and carried forward

v0.0.3 fixes three of the twelve defects v0.0.2 documents, and introduces five of its own.

#The Shape of the Three Releases

v0.0.1 — the complete one

One C++17 translation unit, built to understand backpropagation by implementing every piece of it. It is also the most capable release: mini-batching with shuffling, L2, inverted dropout, a validation split, seeded reproducibility, multi-class softmax fused with cross entropy, and bit-exact model persistence. 112 checks pass, CI runs them on three compiler/OS combinations, and the backward pass is verified against central differences to roughly 10^-11. What it is not is fast: every neuron owns its own weight vector, so the hot path chases a pointer per neuron and allocates inside the training loop.

v0.0.2 — the rewrite

The Neuron class is deleted and its state folded into flat, layer-contiguous vectors; enum dispatch becomes raw function pointers; every buffer is sized once in the constructor. Forward, backward and the update allocate nothing. Python bindings arrive with it. The trade is scope, and it is a steep one: mini-batching, dropout, L2, the validation split, save/load, multi-class and trainable softmax were all left behind. That is why the version number went down — it marks an architecture reset, not an increment.

v0.0.3 — the narrowing

Same layout, different arithmetic over it. Every scalar halves in width so twice as much fits in cache; the unroll hints become OpenMP simd directives; gradients accumulate, so mini-batching returns. Two v0.0.2 defects are fixed and a third is partly addressed. The engine compiles and trains — but python/ and tests/ were never touched, so the release has no automated verification at all.

Two threads run through the sequence. Each release removes an indirection the previous one paid for on every neuron on every pass: v0.0.2 removes the object and the switch, v0.0.3 removes half of every scalar and asks the compiler for vector instructions. On verification the sequence dipped and recovered: v0.0.1 ships a passing suite and CI, v0.0.2 ships a suite with known failures, and v0.0.3’s suite was unported until it was brought back to 91 passing checks — clean under AddressSanitizer and UBSan — which closed eight of the twelve defects its reference page had listed. In-tree CI is now the highest-value work outstanding.

#Where Each Version Lives

#Version Numbering

Patch

Bug fixes and documentation corrections. No API change.

Minor

New features added in a backward-compatible way — a new activation, a new loss, a new keyword argument.

Major

Changes that break existing callers. While the major version is 0, the minor version may also break callers.

These releases do not follow that scheme, and the departure is deliberate. v0.0.2 removed features v0.0.1 had — a smaller API than its predecessor, not a larger one — so the number marks an architecture reset rather than an increment. v0.0.3 restores one of those features and adds none. The intent recorded in CHANGELOG.md is that the first feature-complete release after the rewrite will be numbered 0.2.0.

#Publishing a New Version

Run the helper from the repository root. It copies the newest version's pages into a new directory, restamps the version throughout, marks older versions as superseded, and regenerates the release table and the version selectors on every page.

python docs/newversion.py 0.0.4

Then stamp the version in the places the script deliberately does not touch, and rebuild:

0.0.4/pyproject.toml                 version = "0.0.4"
0.0.4/python/gradiex_module.cpp      m.attr("__version__") = py::str("0.0.4")
0.0.4/CHANGELOG.md                   a new entry

cd 0.0.4 && pip install -e .