Back to the shelf
Read
Self-Refine: Iterative Refinement with Self-Feedback
Madaan et al.
Self-Refine, introduced by Madaan et al., is an iterative self-improvement approach in which a single large language model generates an initial output, provides specific actionable feedback on that output, and then refines the output based on its own feedback, cycling without any additional training or supervised data.
Key takeaways
- Self-Refine uses a single LLM as generator, feedback provider, and refiner simultaneously, requiring no additional training data, reward models, or reinforcement learning.
- The method is evaluated across 7 diverse tasks including dialogue response generation, code optimization, math reasoning, sentiment reversal, and constrained generation, using GPT-3.5, ChatGPT, and GPT-4 as base models.
- Across all evaluated tasks, Self-Refine outputs are preferred by humans and automatic metrics over single-pass generation, improving average task performance by approximately 20% absolute.
- Feedback quality is critical to performance: specific, actionable feedback consistently outperforms generic feedback, which in turn outperforms no feedback, as demonstrated in ablation experiments on Code Optimization, Sentiment Reversal, and Acronym Generation.
- In Code Optimization, GPT-4 with Self-Refine improves the optimization rate from 27.3% to 36.0%, an absolute gain of 8.7%, while Dialogue Response preference scores improve from 25.4% to 74.6% with GPT-4.
- Self-Refine shows diminishing returns with additional iterations, with most gains occurring in early feedback-refine cycles, and does not consistently benefit weaker models such as Vicuna-13B, which struggle to follow the refinement format.
Reflections
No notes on this one yet. I add reflections as I finish or revisit a book.