Version 0.0.2

GradieX — Python Library Reference

The gradiex module is a pybind11 extension wrapping the C++ engine described in the C++ Library Reference. It is a single compiled module with one class.

#Installation

Requires a C++20 compiler, Python 3.9 or later, and pybind11. From the repository root:

pip install .              # normal install
pip install -e .           # editable, for development

NumPy is not required to build, but the module returns NumPy arrays, so it is needed in practice. The engine itself is header-only and is compiled directly into the extension module; nothing else is linked.

To build a redistributable wheel:

pip wheel --no-deps -w dist .

#Quick Start

import numpy as np
import gradiex

X = np.array([[0., 0.], [0., 1.], [1., 0.], [1., 1.]])
Y = np.array([[0.],     [1.],     [1.],     [0.]])

net = gradiex.Network(learning_rate=0.5)
net.add_hidden(8, input_size=2, activation="tanh", init="xavier")
net.add_output(1, activation="sigmoid", loss="mse")
net.build()

history = net.train(X, Y, epochs=4000, shuffle=True, seed=42)

print(history[0], "->", history[-1])     # 0.30554 -> 0.000042
print(net.predict([1., 0.]))             # [0.9932]

A network is declared, then built. Layers are added first; build() constructs the engine and initializes the weights. Nothing can run before build(), and no layers can be added after it.

#Module Contents

gradiex.__version__

Version string, e.g. "0.0.2".

gradiex.__author__

Author string.

gradiex.activations() → list[str]

Names accepted by the activation argument, sorted.

gradiex.losses() → list[str]

Names accepted by the loss argument, sorted. Includes the short aliases.

gradiex.initializers() → list[str]

Names accepted by the init argument, sorted.

gradiex.Network

The only class. See below.

#class gradiex.Network

#Constructor

Network(learning_rate=0.01)

Creates an empty network. learning_rate must be greater than zero, or ValueError is raised.

#Declaring Layers

add_hidden(width, input_size=None, activation="relu", init="he")

Appends a hidden layer of width neurons.

input_size is required only for the first layer. On later layers it defaults to the previous layer's width, and if given explicitly it must equal that width.

Must be called before add_output().

add_output(width, input_size=None, activation="sigmoid", loss="mse", init="he")

Adds the output layer. May be called exactly once, and must come after all hidden layers. loss selects the loss function used by loss(), train(), and the backward pass.

build()

Constructs the underlying engine and initializes all weights and biases. Raises RuntimeError if no output layer was added, or if already called.

initialize()

Alias for build().

#Running the Network

forward(x) → numpy.ndarray

Runs one forward pass and returns the output activations as a fresh 1-D float64 array of length output_size.

x must be 1-D with exactly input_size elements.

predict(x) → numpy.ndarray

Alias for forward(). The engine has no dropout or other train-only behaviour, so the two are identical.

backward(y) → None

Back-propagates one target and immediately applies an SGD update to every layer. forward() must have been called first, because the backward pass reads the cached activations from that call.

y must be 1-D with exactly output_size elements.

loss(x, y) → float

Runs a forward pass on x, then returns the configured loss against y. Note that this does perform a forward pass, so it overwrites the cached activations.

train(X, Y, epochs=100, shuffle=False, seed=None, verbose=False, log_every=0) → list[float]

Trains by per-sample stochastic gradient descent and returns the mean loss for each epoch, computed with the configured loss function.

Argument	Meaning
X	2-D array, shape (n_samples, input_size)
Y	2-D array, shape (n_samples, output_size)
epochs	Number of passes over the whole dataset. Must be >= 0.
shuffle	Reshuffle the sample order every epoch
seed	Seed for the shuffle only. None draws from random_device. This does not make weight initialization reproducible — the engine offers no control over that.
verbose	Print progress to standard output
log_every	Print every nth epoch. 0 picks epochs / 10. Ignored unless verbose is true.

Every row of X and Y is length-checked before training starts, so a badly shaped dataset fails immediately rather than midway. An empty dataset, or epochs=0, returns an empty list and does nothing.

The global interpreter lock is released for the duration of the training loop, so other Python threads continue to run.

#Inspecting Weights

weights(layer) → numpy.ndarray

Returns a copy of one layer's weights, shaped (width, input_size), so W[n][i] is the weight from input i to neuron n.

Layers are indexed 0 upward through the hidden layers, with the output layer last. Negative indices count from the end, so -1 is the output layer.

The array is a copy; writing to it does not affect the network.

biases(layer) → numpy.ndarray

Returns a copy of one layer's biases, a 1-D array of length width. Same indexing rules.

#Properties

repr() summarises the architecture:

>>> net
<gradiex.Network lr=0.5 2 -> 8[tanh] -> 1[sigmoid] loss=mse>

Network is not copyable. The underlying C++ object owns raw pointers, so copying is disabled rather than left to double-free.

#Activation, Loss and Initializer Names

#Activations

#Losses

#Initializers

Biases always start at zero regardless of the initializer.

Names are used rather than function objects on purpose. The C++ API takes an activation and its derivative as two independent function pointers, so a mismatched pair compiles and trains silently wrong. Selecting by name guarantees a matched pair.

#Array Conventions

• Inputs accept anything NumPy can cast to float64: lists, tuples, and integer or single-precision arrays. Casting is automatic.
• Non-contiguous arrays, such as a slice with a step, are copied as needed.
• All returned arrays are fresh, contiguous, 1-D or 2-D float64 arrays. They never alias the network's internal buffers, so the network cannot be modified by writing to them.
• X and Y for train() must be 2-D even when there is a single feature or a single target. Use Y.reshape(-1, 1) for a scalar target.

#Errors Raised

The C++ engine can reach several states that crash the process or train incorrectly. The binding collects layer specifications first and constructs the engine only inside build(), which makes those states unreachable. Each raises a normal Python exception instead:

Only one output layer is permitted because the engine's backwardPass propagates gradients from outputLayers[0] alone; a second head would train silently incorrectly.

#Differences from the C++ Engine

The binding deliberately does not match the C++ API in two places.

Loss reporting uses the configured loss.

core::Network::computeLoss sums squared error regardless of which loss function the layer holds, so training on binary_cross_entropy and reading the loss from C++ reports a different quantity from the one being minimised. Network.loss() and Network.train() call the layer's actual loss function.

train() is reimplemented.

It does not call core::Network::train. The C++ version prints every epoch unconditionally, visits samples in fixed order, and returns nothing. The binding's version returns per-epoch loss history, supports seeded shuffling, is silent unless asked, and releases the GIL.

Everything else — the forward pass, the backward pass, the weight update, and the numerical behaviour of every activation, initializer and loss — is the C++ engine unchanged.

#Examples

#Binary classification with cross entropy

import numpy as np, gradiex

rng = np.random.default_rng(0)
X = rng.normal(size=(400, 4))
Y = ((X[:, 0] + X[:, 1] > 0).astype(float)).reshape(-1, 1)

net = gradiex.Network(learning_rate=0.05)
net.add_hidden(16, input_size=4, activation="relu", init="he")
net.add_hidden(16, activation="relu")            # input_size inferred
net.add_output(1, activation="sigmoid", loss="bce")
net.build()

hist = net.train(X, Y, epochs=200, shuffle=True, seed=1,
                 verbose=True, log_every=50)

pred = np.array([net.predict(x)[0] for x in X])
print("accuracy", ((pred > 0.5) == (Y[:, 0] > 0.5)).mean())

#Manual forward and backward

For finer control, drive one sample at a time. This is what train() does internally.

net.build()
for epoch in range(1000):
    total = 0.0
    for x, y in zip(X, Y):
        net.forward(x)
        net.backward(y)
        total += net.loss(x, y)      # note: runs another forward pass
    if epoch % 100 == 0:
        print(epoch, total / len(X))

Be aware that loss() performs its own forward pass, so the loop above runs two forward passes per sample. To avoid that, capture the loss from the return value of train() instead.

#Inspecting and saving weights

The engine has no save or load facility. NumPy can stand in for inspection, though there is currently no way to write the weights back:

np.savez("model.npz",
         **{f"W{i}": net.weights(i) for i in range(net.num_hidden + 1)},
         **{f"b{i}": net.biases(i)  for i in range(net.num_hidden + 1)})

#Changing the learning rate during training

for lr in (0.1, 0.05, 0.01):
    net.learning_rate = lr
    net.train(X, Y, epochs=100, shuffle=True, seed=7)

#Tests

python python/test_bindings.py

Must be run from the repository root. The suite covers XOR training, reproducibility of the seeded shuffle, the manual forward/backward loop, NumPy casting, and every error in section 7.

#Troubleshooting

AttributeError: module 'gradiex' has no attribute '__version__'

A directory or file named gradiex earlier on sys.path is shadowing the extension module. This happens most often when running a script from a directory that contains a checkout named gradiex. Check gradiex.__file__: it should end in .so or .pyd.

RuntimeError: call build() before using the network

build() was never called, or an exception interrupted the declaration sequence. Check net.built.

Loss does not decrease

Check, in order: that the initializer is not "zeros"; that the learning rate is not so large that the loss oscillates; that inputs are roughly unit-scale, since the engine does no normalization; and that Y is in the range the output activation can produce — a sigmoid output cannot reach values outside (0, 1).

Compilation fails with errors about <numbers>

The engine requires C++20. Confirm the compiler is recent enough; setup.py requests cxx_std=20.

Results differ between runs with the same seed

Expected. seed controls only the shuffle order. Weight initialization draws from a shared generator seeded from std::random_device and cannot be controlled. See C++ Known Bugs, item 8.

#Not Supported

The binding exposes what the engine provides and adds nothing numerical. So there is no momentum or Adam, no learning rate schedule, no mini-batching, no L2 or dropout, no train/validation split, no early stopping, no model save or load, no reproducible weight initialization, no trainable softmax output, and no convolution, recurrence, GPU or threading.

Single precision is not available either: the engine is double throughout, and all arrays are float64.

See C++ Known Bugs and Limitations for defects inherited from the engine, and TODO.md in the repository root for planned performance work.