A new AI performance test requires models to build data analysis pipelines and handle unfamiliar datasets to solve complex problems. Released by researcher @htihle, the WeirdML v3 benchmark features 11 hand-made tasks where models operate with limited feedback and unspecified goals. GPT 6 Astra leads early results in token efficiency, maintaining a large advantage over other tested models in its ability to produce results with fewer tokens.

The benchmark includes a radio telescope simulation called the TOD Pipeline task, which forces agents to extract weak sky signals from atmospheric emission and system temperature spikes. Models must calibrate the instrument and filter radiometer noise to map data onto the sky. These results continue to update as the benchmark adds more models to its evaluation set.