Rethinking Language Model Evaluation: The Importance of Distribution Analysis
A new study highlights the need for evaluating language models beyond single outputs, emphasizing the significance of understanding the broader distribution of possible completions.