---
type: learning
area: learning
status: learning
date: 2026-05-03
created: 2026-05-03
updated: 1980-01-01
tags:
  - learning
---
## Plan
![[img-20240929-122618-00a280dd.png]]

### Example
![[img-20240929-123432-29c7b4c9.png]]
- Self learning deep learning (unsupervised)  
In auto encoders we use tanh as activation function
### Process plan
![[img-20240929-124620-13400468.png]]
Softmax in resume is a function that makes the highest value in output layer to 1 and all other to 0
### Bias 
is just a constant added to activation function added to weights 

### Training process
![[img-20240929-165104-cdc7f045.png]]

### Over-complete Hidden Layers
Auto Encoder Can be used as feature extraction Tool
Adding more Hidden layer permit all to extract more features, That could make network lazy and don't train.
* Solutions for that:
### Sparce Auto encoders
Hidden Layer > Input Layer.
uses a regularization technique to prevent overfiting.
It introduces an constraint that don't allow to auto encoder to don't use all nodes of hidden layers in train but use some of them in each row of data, on every row of data auto encoder uses a specific number of nodes
### Denoising Auto encoders
A stochastic type
![[img-20240929-172128-70f43d89.png]]
Change value of some data to 0 randomly
### Contractive Auto Encoders
![[img-20240929-172425-c086880e.png]]
### Stacked auto encoders
more layers of encoding
![[img-20240929-173115-22a20242.png]]
### Deep auto encoders
![[img-20240929-173317-596bedd4.png]]