💻Deep Learning (DL)

Neural networks, Transformer architecture (basis of GPT), working with images and text. Multi-layered networks

Article body and graph labels may still appear in Russian where English translations have not been added yet.

📖5 min read📊Level 5🗺️10 subtopics📅April 16, 2026

Loading map...

Convolutional Neural Networks (CNN)
For images: convolutional layers detect patterns (edges → textures → objects). Notable models include LeNet (1998), AlexNet (2012), and ResNet (2015). Applications include face recognition and medicine.
Recurrent Neural Networks (RNN/LSTM)
Used for sequence processing: text, time series, audio. RNNs pass hidden states between steps but suffer from vanishing gradients. LSTM (Hochreiter, Schmidhuber 1997) uses gates (forget, input, output) to control information flow. GRU (2014) is a simplified version, replaced by transformers for NLP (2018+)
Recurrent Neural Networks (RNN)
Recurrent Neural Networks for sequences. Hidden state: memory of the past. Problem: vanishing/exploding gradients. LSTM (1997): gates (forget, input, output), long-term memory. GRU: simplified LSTM. Bidirectional: context from both sides. Applications: NLP, time series, music
Recurrent Neural Networks (RNN)
Designed for sequences, they have memory of previous inputs. The vanishing gradient problem is addressed by LSTM (1997) and GRU. Applications include text, speech, and time series.
Transformer Architecture
Introduced an attention mechanism instead of recurrence (2017, 'Attention Is All You Need'). It allows for parallelization and is the foundation of GPT, BERT, and T5, revolutionizing NLP and now vision.
Backpropagation
A neural network training algorithm rediscovered by Rumelhart, Hinton, and Williams (1986). It involves a forward pass, loss calculation, backward pass (using the chain rule), and weight updates (gradient descent). Optimizers include SGD and Adam (2014), with challenges like local minima and gradient issues
Regularization in DL
Techniques to combat overfitting. Dropout (Hinton 2012) randomly "turns off" neurons during training (50%). Batch Normalization (2015) normalizes layer activations. Data Augmentation enhances images (rotations, reflections, cutout). L1/L2 regularization of weights and early stopping based on validation metrics
Convolutional Neural Networks (CNN)
Convolutional Neural Networks for images. Layers: convolution (filters), pooling (compression), fully connected. Hierarchy of features: edges → textures → parts → objects. LeNet (1998), AlexNet (2012, ImageNet), VGG, ResNet (skip connections), EfficientNet. Transfer learning: pretrained features
Attention Mechanism
Attention: weighted sum, focusing on relevant parts. Query-Key-Value: Q·K → weights → V. Bahdanau attention (2014): seq2seq translation. Self-attention: attention to itself. Multi-head attention: parallel attention. Basis of Transformer (2017). Enabled scaling of LLM
Backpropagation
Neural network training algorithm (1986, Rumelhart). Computes gradients of the loss function across all weights using the chain rule. Basis of gradient descent. Requires differentiability

Глубокое обучение - важная область знаний.

💻Deep Learning (DL)

Map outline

🔗Learn more