---
type: learning
area: learning
status: learning
date: 2026-05-03
created: 2026-05-03
updated: 1980-01-01
tags:
  - learning
---
2 Parts:
Encoder, Decoder

Encoder:
- Multiple layers 
- reading and processing 
- generate context rich text presentation
- uses self-attention and feed-forward

Decoder:
- Generat output sequence based on input sequence
- Order is crucial
Attention mechanism 
- used to highlight or focus on important tokens and their relationship imporve text gen
self attention 
- weighting token importane