Natural Language Processing

Preprocessing

• Tokenization

Architectures

• Embedding Model
  • Word Embedding
  • Text Embeddings I
• RNN
  • Recurrent Neural Network
• Transformer
  • Attention Mechanism
  • Transformer
  • Contextual Word Embedding
  • Multi-Head Attention Variants
  • [Positional Embedding / Encoding]
  • FlashAttention
  • [FlashAttention-2]

LLM

• GPT-1, GPT-2 and GPT-3
• GPT Variants: Codex and InstructGPT
• Early Open Source LLM
• CodeGen and FLAN
• LLM: 2022 Jan - 2023 Jun

Multimodal LLM

• Vision Language Pre-Training
• Layout Language Model
• [Visual Instruction Tuning]

LLM Training, Tuning and Deployment

• LLM Scaling Law
• Efficient Training of LLM
• Parameter Efficient Fine-Tuning
• Instruction Tuning
• [Instruction Tuning II]
• Quantization: 8-bit
• [Quantization: 4-bit]

Prompt

• Prompt Engineering
• Prompt Tuning

Evaluation

• Measuring Quantities of Information
• NLP Evaluation
• LLM As A Judge
• LLM Benchmark 2018-2021
• LLM Benchmark 2022-2023