---
type: learning
area: learning
status: learning
date: 2026-05-03
created: 2026-05-03
updated: 1980-01-01
tags:
  - learning
---
Plan of Attack
![[img-20240924-210903-677a4d6b.png]]

NN and difference between them and where to use each one
![[img-20240924-211046-afd1a54a.png]]

Weights are the same all the  the after training and are log term memory of the brain (temporal lobe)

CNN are (Occipital Lobe)

RNN is 
![[img-20240925-175035-b3c197cb.png]]

the Hidden Layers are in blue and the new arrow not means that hidden layers give output to output layer but also to it self. 

![[img-20240924-212756-2e5b0232.png]]
Each circle means a list of layer and neurons.
neurons are connected to them selfs in time each neuron remember what happened before

Examples of some RNNs structure and use cases
![[img-20240924-213717-f1b74b5f.png]]
 
### The vanishing Gradient Problem
![[img-20240924-215330-d59db8cc.png]]

Problem description watch video
![[img-20240924-232124-98d937fa.png]]

In lstm Wrec = 1  ana ths fix the prob with vanishing and exploding gradient
![[img-20240926-120611-8e29a7ef.png]]

C memmory
H output
X input
all of these are vectors
![[img-20240926-121838-28b24d0d.png]]
X in pink is an operation and it's called a valve it's opened or closed based on the result of sigma operation (sigmoid activation function) 
![[img-20240926-122130-aa7925fb.png]]
This equivalent of +
![[img-20240926-122306-10044074.png]]

- Explanation of each element

### Lstm Variations
- Standard LSTM
![[img-20240926-125802-cf484edd.png]]

- Variation 1
![[img-20240926-125907-8bb04d2e.png]]
- Variation 2
![[img-20240926-125949-f0753d44.png]]
- Variation 3
![[img-20240926-130047-eceb95be.png]]