Natural Language Processing
Preprocessing
Architectures
- Embedding Model
- RNN
- Transformer
- Attention Mechanism
- Transformer
- Contextual Word Embedding
- Multi-Head Attention Variants
- [Positional Embedding / Encoding]
- FlashAttention
- [FlashAttention-2]
LLM
- GPT-1, GPT-2 and GPT-3
- GPT Variants: Codex and InstructGPT
- Early Open Source LLM
- CodeGen and FLAN
- LLM: 2022 Jan - 2023 Jun
Multimodal LLM
- Vision Language Pre-Training
- Layout Language Model
- [Visual Instruction Tuning]
LLM Training, Tuning and Deployment
- LLM Scaling Law
- Efficient Training of LLM
- Parameter Efficient Fine-Tuning
- Instruction Tuning
- [Instruction Tuning II]
- Quantization: 8-bit
- [Quantization: 4-bit]