AI Safety & Alignment
cs.AI · 5 papers
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
Two-Domain All Avg: 47.92%Shi-Yu Tian, Yong Liu +11
Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) by routing domain-specific data to specialized teachers, integrating their capabilities through…
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
FID: 1.73Keyu Tian, Yi Jiang +3
We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or…
Are Emergent Abilities of LLMs a Mirage?
BIG-Bench: > 92%Rylan Schaeffer, Brando Miranda, Sanmi Koyejo
Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes…
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
pre-training speedup: 7xWilliam Fedus, Barret Zoph, Noam Shazeer
In deep learning, models typically reuse the same parameters for all inputs. Mixture of Experts (MoE) defies this and instead selects different parameters for each incoming…
An Image is Worth 16x16 Words: Vision Transformers
ImageNet-1K: 88.55%Alexey Dosovitskiy, Lucas Beyer +10
While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision,…