Architecture Transformer MoE Linear Attention FFN Components Tokenizer Norm RoPE KV Cache Activation Functions Model Family BERT GPT DeepSeek Infra Quantization PyTorch FlashAttention DeepSpeed Megatron-LM Training Epoch vs Batch Emergence Optimizer Misc Agent