Researchers at Google DeepMind developed a framework called Dream-RSI that enables AI agents to refine their problem-solving strategies by simulating past discovery runs. The system creates a replay simulator from historical data, allowing agents to test thousands of exploration policies offline to decide where to branch or when to stop without the cost of live rollouts. In GPU kernel tasks, this approach increased performance by 2.09x for specific operations and cut agent calls in Lasso tasks by 162x compared to SimpleTES.

This meta-layer optimization improves the search harness rather than updating the base model weights. Experiments across algorithm design and mathematical optimization showed that this method outperformed traditional prompt-based summaries, which often introduced biases that limited exploration. The framework achieved performance targets for VGG16 and LayerNorm while requiring 2.43x and 1.79x fewer generations, respectively.