Latest

Notes from writing an eval loop

Jun 15, 2026 · 5 min read

Yomimono's generator prompt takes a list of allowed grammar patterns and vocabulary, and asks the model to write a short Japanese reading passage. It's asked to write a real story with a beginning, middle, and end, and characters that have clear motivations. The constraints are tight. The creative ask isn't. The problem I noticed was the model didn't always respect the constraints. It would reach…