Deep Learning Lab

Build neural networks from scratch. Explore 3D error surfaces, optimizers, backpropagation, overfitting and convolutions.

About the Deep Learning Lab

The Deep Learning Lab follows the same path the field took: from a single threshold neuron, through the XOR problem that stalled research for years, to backpropagation, modern optimisers and convolutions, and finally a language model predicting its next token.

You can rotate a three-dimensional loss surface and watch eight optimisers race to the bottom of it, train a small network with your own choice of activation, dropout and L2 penalty, or follow an error signal backwards through every weight. Nothing is hidden behind a library call.

It goes with the Deep Learning course, and almost every experiment links to the lesson that explains it. It suits students starting deep learning, and practitioners who want a feel for why one optimiser or initialisation behaves differently from another.

Experiments in this lab

Thresholding neuron: Logic gates from a threshold and a line.
Perceptron learning: Watch the weights change after every mistake.
The XOR problem: One line cannot separate XOR. Two layers can.
Sigmoid versus step: Why a smooth neuron makes learning possible.
Taylor series: Approximate a curve by a line or polynomial near a point.
Error surfaces (3D): Rotate a loss surface and find its valley.
Tower functions (3D): Build any shape from sigmoid towers.
Entropy and loss: How cross-entropy responds to confidence.
Backpropagation flow: Follow a signal forward, then the error backward.
Optimizer playground: Race eight optimisers on the same loss surface.
Overfitting lab: Bias, variance, L2 and early stopping on a curve fit.
Network lab: Train a network with any optimiser, dropout, L2 and more.
Activations and init: Activation shapes, maxout, and signals in deep networks.
Convolutions: Slide a filter over an image.
LLM inference: Watch a language model predict the next token.

Deep Learning — The course this simulator goes with
Transformer Lab — Tokenize with BPE, watch attention heads, rotate positions with RoPE, size a KV cache, route tokens to experts and quantize weights.
RAG Lab — Chunk a handbook, search it by meaning, fuse BM25 with dense retrieval, rerank, measure recall and NDCG, and keep the index fresh.
Linear Algebra Lab — Drag vectors to see a span collapse, warp the grid with a matrix, hunt eigenvectors by hand, run Gram–Schmidt step by step, and watch a circle become an ellipse under the SVD.
Machine Learning Lab — Find principal components, run k-means and EM step by step, tame overfitting with ridge and lasso, and compare kNN, trees, naive Bayes, SVMs and boosting.
Image Processing Lab — Sampling, filtering, Fourier transforms, segmentation, SIFT and HOG, applied to a live image you can change.
All simulators