Language Models are Few-Shot Learners

基本信息

• Title: Language Models are Few-Shot Learners
• Source type: paper
• Related topic notes: Decoder-only Transformer, Training Objective, GPT

TODO

• 阅读论文原文，整理 GPT-3 的 decoder-only language modeling 路线、in-context learning 现象和规模化设置。
• 回填 decoder-only Transformer 如何把多任务统一为 prompt + continuation。
• 对照现代 instruction tuning / post-training，区分 base LM 能力与 assistant 行为塑形。