Shrinking the Generation-Verification Gap with Weak Verifiers

Verifiers can improve language model capabilities by scoring and ranking responses from generated candidates. Currently, high-quality verifiers are either unscalable e.g., humans or limited in utility e.g., tools like Lean. While LM judges and reward models have become broadly useful as...