OpenAI tests agents on messy wet-lab evidence with BioBasicsBench

OpenAI and enzyme-design company Ligo have introduced BioBasicsBench, a 100-task benchmark designed to test AI agents on routine wet-lab analysis rather than polished figures from scientific papers. The tasks use raw evidence such as gel images, colony plates, microscopy stacks, sequencing files, instrument exports, handwritten notes and protocol videos.

The benchmark covers protein methods, imaging and cytometry, quantification and assays, nucleic-acid methods and molecular cloning. Agents must interpret measurements, identify faulty controls and recognize when the evidence does not support a firm conclusion. Scientists created the tasks from their own experiments, and each one received internal and external PhD-level review.

OpenAI evaluated seven model families across 37 model-and-reasoning settings, with five attempts per task and 18,500 results in total. At xhigh reasoning effort, GPT-6 Astra earned 66.7% of the available credit, compared with 62.0% for GPT-6.1 Sol and 43.8% for GPT-5.6 Sol. OpenAI attributes much of Astra’s improvement to stronger multimodal handling.

The results also show why the benchmark is difficult. Astra sometimes let assumptions override experimental controls, discarded observations that contradicted its preferred explanation and overcounted colonies by about 12% in one plate image.

A higher benchmark score does not mean the models are ready to make unsupervised laboratory decisions. BioBasicsBench instead provides a more realistic measure of whether agents can help scientists troubleshoot everyday experiments while recognizing ambiguity and failed evidence.