Linear Algebra Cheat Sheet

 

Common Convolutional Neural Networks

1. LeNet - 5 - 1998

2. AlexNet - 2012

3. VGG - 16 - 2015

4.  ResNet

Adam Algorithm

1. Adam Algorithm

2. Parameters

Convolution Layer

1. The idea of convolution layer

# of parameter dimensions:

# dimension reduction by 1 convolution operation:

2. The idea of ConvNet

some notes:

1. the shape of the layers go smaller and smaller.

2. the number of channels go larger and larger.

3. the final results of ConvNet will be flatten and then feed into a NN layer. 

3. The idea of Pooling Layer (Max Pooling)

4. Putting it together

Neural Network Forward and Backward Compute (N layers)

1. Forward Propagation

2. Backward Propagation

3. Put it together

Neural Network Forward and Backward Compute (2 layers)

1: Forward Compute with One Example

2: Forward Compute with Vectorizing across multiple examples

3. Gradient for Backward Compute

4. Formula for Backward Compute

5. To understand why the partial derivatives are calculated, check out the book here: http://www.deeplearningbook.org/  

or check the video here:

https://www.coursera.org/learn/neural-networks-deep-learning/lecture/6dDj7/backpropagation-intuition-optional

ImageNet

1. What it is?

More than 14 million[1][2] images have been hand-annotated by the project to indicate what objects are pictured and in at least one million of the images, bounding boxes are also provided.[3] ImageNet contains more than 20,000 categories[2] with a typical category, such as "balloon" or "strawberry", consisting of several hundred images.

2. History

AI researcher Fei-Fei Li presented their database for the first time as a poster at the 2009 CVPR.

3. ImageNet Large Scale Visual Recognition Challenge (ILSVRC)

GoogleLeNet-v4 is the winner until now: https://www.google.com/search?q=imagenet+accuracy+over+time&tbm=isch&source=univ&sa=X&ved=2ahUKEwiI_7yQ3tLmAhVFSX0KHdjICC8QsAR6BAgFEAE&biw=1920&bih=897#   

https://machinelearningmastery.com/introduction-to-the-imagenet-large-scale-visual-recognition-challenge-ilsvrc/