Fundamentals of deep learning: a step-by-step guide (Note: this manuscript is a preprint of a draft book, and I hope it can benefit faculty, students or engineers who want to start a journey of "deep learning". Comments or feedbacks can be sent to weidong.kuang@utrgv.edu. Your comments and feedbacks are invaluable for me to improve it.) Content Preface (pdf) (pp.4) (updated on 2/23/2023) Chapter 1 Introduction to Deep Learning (pdf) (pp.19) (19) (2/23/2023) 1.1 Introduction 1.2 Types of Machine Learning 1.3 Data Representation in Machine Learning 1.4 Overview of Deep Learning 1.5 Resources for Deep Learning References Exercises Chapter 2 A Brief Mathematical Review (pdf) (pp.51) (70) (2/23/2023) 2.1 Vector and Matrix 2.2 Matrix Calculus 2.3 Probability and Statistics 2.4 Gradient-Based Optimization Summary and Further Reading References Exercises Chapter 3 Linear Regression (pdf) (pp.25) (95) (4/18/2023) 3.1 Linear Regression with Single Feature 3.2 Linear Regression with Multiple Features 3.3 Linear Models for Regression 3.4 A Probabilistic Interpretation of Linear Regression 3.5 An Example: House Price Prediction Summary and Further Reading Exercises Chapter 4 Classification and Logistic Regression (pdf) (pp.21) (116)(5/2/2023) 4.1 Logistic Regression 4.2 Metrics for Classification Performance 4.3 Implementation of Logistic Regression in Python Summary and Further Reading Exercises Chapter 5 Basics of Neural Networks (pdf) (pp.37) (153) (8/28/2023) 5.1 A Simplest Neural Network: a Logistic Regression Unit 5.2 From Regression to Neural Networks 5.3 Neural Network Representation: Feed-forward Propagation 5.4 Activation Functions 5.5 Network Training: Backward propagation 5.6 Multi-class Classification: Softmax and Cross Entropy Loss 5.7 Practice in Python Summary and Further Reading Exercises Chapter 6 Practical Considerations in Neural Networks (pdf) (pp.56) (209) (8/28/2023) 6.1 Multiple-Layer Neural Networks 6.2 Generalization and Model Selection 6.3 Regularization 6.4 Weight Initialization 6.5 Mini-batch Gradient Descent 6.6 Normalization 6.7 Adam Optimization 6.8 Gradient Checking 6.9 Examples in Python Summary and Further Reading Exercises Chapter 7 Introduction to PyTorch (pdf) (pp.37) (246) (8/28/2023) 7.1 Why PyTorch? 7.2 Tensors 7.3 Data Representation using Tensors 7.4 Linear Regression using PyTorch 7.5 Neural Networks using PyTorch 7.6 An Example by PyTorch: Finger Signs Summary and Further Reading Exercises Chapter 8 Convolutional Neural Networks (pdf) (pp.32) (278) (9/8/2023) 8.1 Architecture of Convolutional Neural Networks 8.2 Convolutional Layer 8.3 Pooling Layer and Fully Connected Layer 8.4 Backpropagation in CNNs 8.5 Batch Normalization for CNNs 8.6 Implement CNNs in PyTorch Summary and Further Reading Exercises Chapter 9 Classical CNN Architectures (pdf) (pp.29) (307) (9/18/2023) 9.1 Datasets 9.2 AlexNet 9.3 VGG: Networks using Blocks 9.4 Network in Network 9.5 GoogLeNet 9.6 ResNet 9.7 Pretrained Models Summary and Further Reading Exercises Chapter 10 Object Detection -- YOLO (pdf) (pp.46) (353) (12/11/2023) 10.1 Introduction 10.2 YOLO(v1) 10.3 YOLO(v2) 10.4 YOLO(v3) 10.5 Implementation of YOLO(v3) using Pre-trained Model 10.6 A Metric for Object Detection mAP Summary and Further Reading Exercises Chapter 11 Introduction to Probabilistic Generative Models (pdf) (pp.30) (383) (1/16/2024) 11.1 Generative Models with Latent Variable 11.2 EM Algorithm 11.3 Variational Auto_Encoder 11.4 VAE Implementation on MNIST Dataset in PyTorch Summary and Further Reading Exercises Chapter 12 Generative Adversarial Networks (pdf) (pp.43) (426) (3/19/2024) 12.1 Mathematical Description of Original GAN 12.2 Framework of GAN Implementations 12.3 GAN Variants 12.4 Example: DCGAN on MNIST Summary and Further Reading Exercises
The following chapters will be updated soon and more chapters will be added. Please check frequently (more chapters are coming soon) Appendix A Jupyter Notebook and PyTorch Installation (doc)
Note: if you need the copies of python files in the manuscript, please send a request to weidong.kuang@utrgv.edu
|