The model lab · Part 5
How do neural networks recognize digits?
Meet Doodle-525: draw any digit and watch its hidden layer find the strokes before it names the number.
Doodle-525
Model card- Type
- Two-layer feed-forward neural network (a tiny multi-layer perceptron)
- Parameters
- 525: 448 pixel-to-stroke weights + 7 stroke biases + 70 stroke-to-digit weights
- Layers
- 2 (64 pixels → 7 stroke detectors → 10 digits)
- Input
- 64 numbers: the 8×8 grid, 1 where you drew ink and 0 where you didn't
- Output
- 10 probabilities, one per digit 0-9, summing to 100%
Same inputs as Doodle-64, eight times the parameters, and one hidden layer, the single upgrade that "deep" learning repeats hundreds of times in a billion-parameter LLM.
Draw a digit: 0-9
Click a pixel, or drag across the grid to draw a stroke at once. On a phone, draw with your finger. A two-layer network, 64 inputs, 7 stroke detectors, 10 outputs, will read it.
Layer 1: seven stroke detectors
Each hidden neuron watches one stroke, a bar or a line, and its weights are drawn below in the same 8x8 layout as the drawing grid. Colored cells are the pixels that neuron cares about; everything else has weight zero. Draw something and watch each detector report how much of its stroke it sees.
Layer 2: combining strokes into digits
The output layer's 70 weights, one row per digit. Green means "this digit expects that stroke", red means "that stroke is evidence against it". An 8 wants every stroke; a 1 only wants the two right-hand lines, and every other stroke argues against it.
| Digit | Top bar | Upper-left line | Upper-right line | Middle bar | Lower-left line | Lower-right line | Bottom bar |
|---|---|---|---|---|---|---|---|
| 0 | + | + | + | − | + | + | + |
| 1 | − | − | + | − | − | + | − |
| 2 | + | − | + | + | + | − | + |
| 3 | + | − | + | + | − | + | + |
| 4 | − | + | + | + | − | + | − |
| 5 | + | + | − | + | − | + | + |
| 6 | + | + | − | + | + | + | + |
| 7 | + | − | + | − | − | + | − |
| 8 | + | + | + | + | + | + | + |
| 9 | + | + | + | + | − | + | + |
Watch your drawing run through the network
The same animation as the image classification page, one layer deeper: your 64 pixels light up the stroke detectors, and the detectors vote on all ten digits.
Each colored edge feeds one stroke detector; press run and watch your lit pixels wake up the detectors whose strokes you drew. Then green edges carry each firing detector's vote toward the digits that use that stroke, and red edges carry its vote against the digits that don't. The digit collecting the most evidence wins.
Prediction
How this classifier works
The 3-vs-E classifier needed only one layer, because single pixels were enough to separate two shapes. Ten digits share almost every pixel: an 8 contains a 3, a 9 contains a 7, so no single pixel can vote cleanly for one digit. The fix is a hidden layer that finds bigger patterns first:
- Layer 1 finds strokes. Each of the seven hidden neurons sums the pixels of one bar or line and fires when most of its stroke is drawn. It knows nothing about digits. It only knows its stroke.
- Layer 2 combines strokes. Each digit output adds +1 for every firing detector the digit uses and −1 for every firing detector it doesn't. A quiet detector votes in reverse: no bottom bar is real evidence for a 4 and against an 8.
- Softmax picks a winner. The ten scores become probabilities that add up to 100%, so the network always ranks every digit.
Try drawing a digit with a stroke missing, or add an extra stroke, and watch the probabilities shift between the digits that share those strokes. That's the whole idea of depth: early layers find parts, later layers reason about combinations of parts. It's exactly what real vision models do, with millions of learned weights instead of 518 hand-built ones.
Meet Doodle-525
Doodle-525 is the second model in our lab, and its model card shows what one extra layer costs and buys. It reads the same 64-pixel grid as Doodle-64, but instead of mapping pixels straight to an answer, it spends 448 weights (plus 7 biases) turning pixels into 7 stroke detections, then 70 more weights turning strokes into 10 digit scores. That comes to 525 for a 10-way choice, where Doodle-64 needed 64 for a 2-way choice.
That budget is the whole story of neural network design: more possible answers and subtler distinctions demand more parameters, and layering lets the parameters share work. The 7 stroke detectors are reused by all 10 digits, the way an 's early layers are reused by every sentence it will ever read.
Why one layer stops being enough
The 3-vs-E classifier got away with a single layer because one pixel could settle the argument: ink on the left edge means E, ink on the right curve means 3. With ten digits that trick collapses. An 8 contains every pixel of a 3, a 9 contains every pixel of a 7, so no single pixel is evidence for exactly one digit.
The fix is the most important idea in deep learning: don't go straight from pixels to answers. First recognize parts, then reason about combinations of parts. That "first" step is a hidden layer.
Layer 1 finds strokes first
The demo below has seven hidden neurons, and each one is a stroke detector: a top bar, a middle bar, a bottom bar, and four vertical lines. Each detector's weights cover just the pixels of its stroke. You can see them as seven mini heatmaps, each in its own color. When you draw most of a stroke, its detector fires.
Notice what these neurons don't know: anything about digits. The top-bar detector fires for a 5, a 7, and an 8 alike. It has one tiny job, done with a handful of weights.
Layer 2 combines strokes into digits
The second layer is where digits exist. Each of the ten outputs holds a recipe: +1 for every stroke the digit uses, -1 for every stroke it doesn't. A firing detector pushes up the digits that want its stroke and pushes down the ones that don't. A silent detector votes in reverse, because a missing bottom bar really is evidence for a 4 and against an 8.
Softmax then turns the ten scores into probabilities that sum to 100%. Draw an 8 and leave out the middle bar: the network's vote swings to 0, exactly the digit whose recipe matches what you actually drew.
This is exactly what deep networks do
Real image models are this demo scaled up. Their early layers learn edge and color detectors, middle layers combine edges into textures and shapes, and late layers combine shapes into "cat ear" or "handwritten 7". Nobody programs those features. finds them, the way our stroke detectors were chosen by hand here.
"Deep" in deep learning just means many hidden layers stacked, each reasoning about the patterns the previous one found. Two layers read seven strokes; a hundred layers can read a photograph.
If you want to see the next rung of that ladder, Doodle-918 does this same job with a second hidden layer that turns strokes into shapes (loops, curves and spines) before naming a digit. It is the same demo, one layer deeper, and it reads every digit at over 99% confidence.
Try it yourself
Use the example buttons to load a clean digit, then vandalize it. Erase one stroke, add another, and watch the stroke detectors switch on and off and the probabilities slide between the digits that share those strokes. Then press play on the network animation to watch the whole forward pass: pixels light detectors, detectors vote on digits.
Put it to use
Next up: What is gradient descent?