The model lab · Part 1
How do neural network weights work?
The tiny dials that make neural networks learn anything.
Interactive Dot Product
A weight is a wire. Drag a wire up to make its input count for more, down to make it count against. The wire thickens, changes color, and the sum on the right moves with it. Drag an input dot the same way to change the number arriving on that wire: the dot brightens, and the sum moves again.
Or set a weight directly
Input values
a1 × w1 = 0.50 × 0.50 = 0.2500
a2 × w2 = 0.50 × 0.50 = 0.2500
a3 × w3 = 0.50 × 0.50 = 0.2500
Total Output
0.7500
Two weights at once make a surface
One weight moving traces a line, which is not worth a chart. Sweep two of them together and you get the whole shape: a surface, with the third input a3 × w3 holding it up from below. Drag to rotate it. The dot is where your weights are right now.
Output here is 0.75. The surface rises 0.50 for every unit of w1 and 0.50 for every unit of w2. Those are the two inputs' values. Set an input to 0 and the surface goes flat in that direction: its weight can no longer do anything.
Rotate it as much as you like: it is always flat. That is not a limitation of this demo, it is what one layer of weights is. Multiplying and adding can tilt a plane but never bend it, so a single layer can only ever draw a straight boundary between its answers. Bending it takes another layer, which is what the next section is for.
What's Happening?
Inputs
Each input is a number (0 to 1 in this example). In a real neural network, inputs might be pixels in an image, words in a sentence, or features of data.
Weights
Weights are "dials" the network learned during training. They control how much each input matters. If a weight is 0, that input doesn't affect the output. If it's high, that input has a big effect. A negative weight flips the sign: that input now argues against the output instead of for it.
The Calculation
For each input, we multiply it by its weight. Then we add all those products together. That sum is the output of this layer.
Why It Matters
During training, the network adjusts millions of weights until they're "just right" for the task. Better weights = better predictions. That's the whole game.
What are weights?
are numbers inside a , like tiny dials on a massive control panel. During training, the network adjusts millions of these dials until it gets good at its job.
Think of them like the volume knob on a speaker. Turn it up, the sound gets louder. In a network, "turn up" a weight and that input has more effect on the output.
How they work: the multiplication
Each input gets multiplied by its weight. A weight of 0 means "this input doesn't matter." A weight of 2 means "pay double attention to this input." So if an input is 0.5 and its weight is 2, the product is 1.0.
Then all those products get added together. That sum is the output of the layer. The equation below is the whole operation, and it repeats billions of times across a model.
Why they matter
A model's knowledge lives in its weights. They encode everything the network learned during : what a dog looks like in pixels, what makes good code, the patterns in human speech. Change the weights, the model's behavior changes.
When you a model, you're tweaking its weights. When someone sells a model, they're selling the weights (and the architecture that uses them).
The training process
Training a model is a loop: (1) Make a guess with the current weights, (2) See how wrong it is, (3) Adjust the weights to be less wrong, (4) Repeat with the next of data.
This happens millions of times. Each tiny adjustment nudges the weights towards better predictions. After weeks or months, the weights are "just right" for the task, and you have a trained model.
A concrete example
Imagine training a model to predict house prices. Inputs might be: square footage, number of bedrooms, location. Weights learn how much each factor matters. Maybe the model learns: "100 extra square feet is worth $15,000, but being one neighborhood over costs $50,000."
Those relative values are encoded in the weights. They're the model's learned understanding of the housing market, baked into numbers.
Advanced: stacking layers
One neuron doing multiply-and-add is neat, but the magic starts when you stack them. The outputs of one layer become the inputs of the next, so the second layer works with patterns the first layer found, and the third layer works with patterns of patterns.
Watch the animation below: values enter on the left, and each layer computes its neurons from the layer before it. Add and remove hidden layers to see the stack change, and hover any hidden neuron to read the pattern it is looking for. This exact flow happens every time you send a message to an AI model. The only difference is scale: thousands of layers, millions of neurons.
Watch a signal travel through a deep network
This network has 4 → 5 → 5 → 2 neurons across 4 layers, and 55 weights in total. Press play and watch each layer compute its values from the layer before it.
- Hidden 1combines the raw inputs into simple features, things like "is this pair of inputs both high?"
- Hidden 2combines those features into patterns. It never sees the inputs, only what layer 1 reported.
That stacking is the only reason depth helps. A single layer can just tilt a flat plane, the one you rotated further up the page. Each extra layer folds that plane again, and enough folds can carve any shape at all, which is why "deep" learning is deep.
Blue lines are positive weights (excite the next neuron), orange lines are negative (dampen it). Thicker = stronger. Each neuron squashes its weighted sum into 0-1, and brighter circles mean values closer to 1. Hit "Random weights" and the same inputs give a completely different answer. The knowledge is in the weights.