Fundamentals of deep learning: a step-by-step guide

(Note: this manuscript is a preprint of a draft book, and I hope it can benefit faculty, students or engineers who want to start a journey of "deep learning". Comments or feedbacks can be sent to weidong.kuang@utrgv.edu. Your comments  and feedbacks are invaluable for me to improve it.)

Content

Preface (pdf) (pp.4) (updated on 2/23/2023)

Chapter 1 Introduction to Deep Learning (pdf) (pp.19) (19) (2/23/2023)

     1.1 Introduction

     1.2 Types of Machine Learning

     1.3 Data Representation in Machine Learning

     1.4 Overview of Deep Learning

     1.5 Resources for Deep Learning

     References

     Exercises

Chapter 2 A Brief Mathematical Review (pdf) (pp.51) (70) (2/23/2023)

     2.1 Vector and Matrix

     2.2 Matrix Calculus

     2.3 Probability and Statistics

     2.4 Gradient-Based Optimization

     Summary and Further Reading

     References

     Exercises

Chapter 3 Linear Regression (pdf) (pp.25) (95) (4/18/2023)

     3.1 Linear Regression with Single Feature

     3.2 Linear Regression with Multiple Features

     3.3 Linear Models for Regression

     3.4 A Probabilistic Interpretation of Linear Regression

     3.5 An Example: House Price Prediction

     Summary and Further Reading

     Exercises

Chapter 4 Classification and Logistic Regression (pdf) (pp.21) (116)(5/2/2023)

     4.1 Logistic Regression

     4.2 Metrics for Classification Performance

     4.3 Implementation of Logistic Regression in Python

     Summary and Further Reading

     Exercises

Chapter 5 Basics of Neural Networks (pdf) (pp.37) (153) (8/28/2023)

     5.1 A Simplest Neural Network: a Logistic Regression Unit

     5.2 From Regression to Neural Networks

     5.3 Neural Network Representation: Feed-forward Propagation

     5.4 Activation Functions

     5.5 Network Training: Backward propagation

     5.6 Multi-class Classification: Softmax and Cross Entropy Loss

     5.7 Practice in Python

     Summary and Further Reading

     Exercises

Chapter 6 Practical Considerations in Neural Networks (pdf) (pp.56) (209) (8/28/2023)

     6.1 Multiple-Layer Neural Networks

     6.2 Generalization and Model Selection

     6.3 Regularization

     6.4 Weight Initialization

     6.5 Mini-batch Gradient Descent

     6.6 Normalization

     6.7 Adam Optimization

     6.8 Gradient Checking

     6.9 Examples in Python

     Summary and Further Reading

     Exercises

Chapter 7 Introduction to PyTorch (pdf) (pp.37) (246) (8/28/2023)

     7.1 Why PyTorch?

     7.2 Tensors

     7.3 Data Representation using Tensors

     7.4 Linear Regression using PyTorch

     7.5 Neural Networks using PyTorch

     7.6 An Example by PyTorch: Finger Signs

     Summary and Further Reading

     Exercises

Chapter 8 Convolutional Neural Networks (pdf) (pp.32) (278) (9/8/2023)

     8.1 Architecture of Convolutional Neural Networks

     8.2 Convolutional Layer

     8.3 Pooling Layer and Fully Connected Layer

     8.4 Backpropagation in CNNs

     8.5 Batch Normalization for CNNs

     8.6 Implement CNNs in PyTorch

     Summary and Further Reading

     Exercises

Chapter 9 Classical CNN Architectures (pdf) (pp.29) (307) (9/18/2023)

     9.1 Datasets

     9.2 AlexNet

     9.3 VGG: Networks using Blocks

     9.4 Network in Network

     9.5 GoogLeNet

     9.6 ResNet

     9.7 Pretrained Models

     Summary and Further Reading

     Exercises

Chapter 10 Object Detection -- YOLO (pdf) (pp.46) (353) (12/11/2023)

     10.1 Introduction

     10.2 YOLO(v1)

     10.3 YOLO(v2)

     10.4 YOLO(v3)

     10.5 Implementation of YOLO(v3) using Pre-trained Model

     10.6 A Metric for Object Detection mAP

     Summary and Further Reading

     Exercises

Chapter 11 Introduction to Probabilistic Generative Models (pdf) (pp.30) (383) (1/16/2024)

     11.1 Generative Models with Latent Variable

     11.2 EM Algorithm

     11.3 Variational Auto_Encoder

     11.4 VAE Implementation on MNIST Dataset in PyTorch

     Summary and Further Reading

     Exercises

Chapter 12 Generative Adversarial Networks (pdf) (pp.43) (426) (3/19/2024)

     12.1 Mathematical Description of Original GAN

     12.2 Framework of GAN Implementations

     12.3 GAN Variants

     12.4 Example: DCGAN on MNIST

     Summary and Further Reading

     Exercises

 

The following chapters will be updated soon and more chapters will be added. Please check frequently

(more chapters are coming soon)

Appendix A   Jupyter Notebook and PyTorch Installation (doc)

 

Note: if you need the copies of python files in the manuscript, please send a request to

            weidong.kuang@utrgv.edu