Large language models now adapt to new tasks, as the ARC-AGI-3 benchmark shows. In a new study, we asked whether they can also improve their own harness for this benchmark. They build harnesses well, but their improvements rarely transfer to games they have not seen. We discuss why and what that means for AI research.