The Deep Learning Crash Course for Beginners by Jason Dsouza on freeCodeCamp is the map of the territory: what DL is, how a network learns, the vocabulary (loss, Adam, epoch) and the three architecture families. This text condenses the course. The math underneath — MatMul, GPU — is in GPU vs TPU; the neuron itself, in What is a neural network? (3Blue1Brown).
AI, ML, Deep Learning
IA
└─ Machine Learning (aprende com dados, não com if/else)
└─ Deep Learning (rede neural com várias camadas ocultas)
Classic ML requires you to choose the features. DL learns the representation directly from raw data (end-to-end). It powers AlphaGo, vehicle vision, diagnosis and LLMs — the same stack you run on Linux with Ollama.
The cycle: forward, error, backward
- Forward: input → hidden layers (weight
w, biasb, activation) → output guess. Initially the weights are random; the guess is garbage. - Loss: a number that measures how wrong it was. The higher, the worse.
- Backprop + gradient descent: the error propagates backward. The chain rule tells how much each weight contributed. The optimizer moves in the direction opposite to the gradient and the error drops in the next round.
Vocabulary that comes up in training
Activation — without nonlinearity, N linear layers = 1 layer. That is why σ exists.
| Function | Output | Use |
|---|---|---|
| Sigmoid | (0, 1) | Binary output; saturates in the middle of the network |
| Tanh | (−1, 1) | Centers at zero; better than σ in the middle, a classic |
ReLU max(0, x) |
[0, +∞) | Default in hidden layers |
| Leaky ReLU | ℝ | Avoids dead neurons (small gradient when x is less than 0) |
| Softmax | probabilities sum to 1 | Multiclass output |
Loss: regression → MSE (penalizes large errors) or MAE (more robust to outliers). Classification → binary / categorical cross-entropy.
Optimizer: a learning rate that is too high oscillates; too low never gets there. SGD uses mini-batches; Momentum dampens; RMSprop/Adagrad adapt per parameter; Adam (momentum + RMSprop) is the industry default.
Parameter the network adjusts itself (weight, bias). Hyperparameter you choose: lr, batch, epochs, architecture.
1 epoch = 1 pass over the dataset. Batch size = how many examples before updating. 1.000 samples / batch 100 = 10 iterations per epoch.
Three ways to learn — and overfitting
- Supervised: pair (x, y). Image with label, price with value.
- Unsupervised: no label. Cluster, autoencoder, PCA.
- Reinforcement: agent, environment, reward. AlphaGo, robot.
Underfit: model too simple, wrong even during training. Overfit: memorized training, breaks on the test.
Regularization that works: Dropout (turns off neurons during training), L1/L2 on weights, data augmentation (rotate, crop, flip), early stopping (stop when validation loss rises).
Three architectures
- MLP / dense: layer by layer, no memory. Tables, classic regression.
- RNN / LSTM / GRU: loop, internal state. Time series, text, server logs. LSTM/GRU have gates to prevent gradients from vanishing in long sequences.
- CNN: kernel slides across the image, feature map + pooling. Vision. (Transformers took over some text tasks; CNNs still rule classic vision.)
The 5 steps of any project
- Data. Garbage in, garbage out. Class imbalance makes accuracy misleading.
- Preprocessing. Nulls, scale [0,1] or z-score, train / validation / test split (~80 / 10 / 10).
- Training. Forward, loss, backprop, epochs.
- Evaluation. Beyond accuracy: precision, recall, F1, confusion matrix — mandatory if the rare class matters.
- Tuning. Grid/random search over hyperparameters, dropout, lr, more layers if needed.
References
- Course: Jason Dsouza — Deep Learning Crash Course for Beginners (freeCodeCamp)
- What is a neural network? (3Blue1Brown, MNIST)
- GPU vs TPU
- Ollama on Linux
Weight, loss, Adam, ReLU, split. With that the crash course fits in your head — the rest is training and watching the validation plot.