Lesson · ⏱ 8 min read

🕸️ Neural networks: layers that learn

Q: What do the 'weights' in a neural network represent?

Learned numbers that decide how much each input matters. Training adjusts the weights. They're what the network 'knows'.

🏠 Real-life example

Think of a factory assembly line for recognising a handwritten digit. The first workers spot simple strokes and edges. The next combine them into curves and loops. The last workers say 'loop on top plus line below: that's a 9'. Each group of workers is a layer, and training teaches every worker what to look for.

💡 In simple words

A neural network is layers of simple units (neurons). Each takes numbers in, multiplies them by learned weights, adds them up and passes the result through an activation function, like ReLU, that adds non-linearity. Stacking many layers ('deep' learning) lets the network learn increasingly complex patterns. Training runs forward (make a prediction), measures loss, then runs backpropagation and gradient descent to update the weights, over many passes (epochs). PyTorch is the most popular library. Convolutional networks (CNNs) specialise in images.

🎬 See how it works

Signals flow left to right: pixels → simple patterns (edges) → bigger patterns (loops) → the answer. Training adjusts every connection's weight.

🪜 Step by step

Input layer: the raw numbers (pixels, word embeddings, features).
Hidden layers: each neuron computes a weighted sum of its inputs and passes it through an activation like ReLU, so the network can learn curves, not just straight lines.
Output layer: a number for regression, or probabilities (softmax) for classes.
Training: forward pass to predict, loss to measure error, backpropagation to compute gradients, optimiser to update weights.
Regularise with dropout, weight decay and early stopping, and use pretrained models whenever possible.

🔑 Key terms

Weight: A learned number that controls how much an input matters.
Activation function: A small function (e.g. ReLU) that lets networks learn non-straight-line patterns.
Epoch: One full pass over the training data.

👀 See it in action

python

import torch.nn as nn
model = nn.Sequential(
    nn.Linear(784, 128), nn.ReLU(),   # 28×28 image → 128 features
    nn.Linear(128, 64),  nn.ReLU(),
    nn.Linear(64, 10)                 # scores for digits 0–9
)

A small 3-layer network for handwritten digits.

⚠️ Common mistakes to avoid

Training a big network from scratch on a small dataset. Fine tune a pretrained model instead.
Not normalising inputs. Neural networks train poorly when inputs have very different scales.

✅ Quick check

What do the 'weights' in a neural network represent?

The size of the dataset
The GPU's memory
Learned numbers that decide how much each input matters

Show answer

C. Learned numbers that decide how much each input matters: Training adjusts the weights. They're what the network 'knows'.

🙋 Questions learners ask

Why are they called 'deep'?

Because they have many layers stacked. Each layer builds more abstract features from the previous one: edges, shapes, then objects.

PyTorch or TensorFlow?

PyTorch is the most common choice in research and most new projects today. The concepts transfer either way.

📌 Remember

Layers of weighted sums plus activations, trained by backprop.

📖 Words from this lesson (AI dictionary)

Learn this as part of a full course

Track your progress, take the quick checks and follow the lessons in the right order. This lesson is part of:

ML Engineer · AI Research Engineer

Continue in the course