A Mechanistic Fallibilist

Much of what I see being built around AI and labs looks like a new kind of contract research organization (CRO). A customer brings a question or a protocol; the service runs experiments and returns data. Models and robots can make that service faster and more reliable. But the customer still does much of the work of deciding what is worth investigating.

Cloud labs show how useful this can be. Emerald Cloud Lab lets researchers specify experiments remotely and have them run in an automated facility. Access to a lab no longer has to mean standing at its bench.

The larger ambition is to let the lab carry more of the investigation itself: decide which test would help, notice when the result is misleading, and change direction when an explanation fails. That is a much bigger step than taking a protocol and running it well.

I spent ten months training models with reinforcement learning on tasks where success was hard to verify. A recurring problem was that a model could get a higher score without doing a better job. Physical experiments seem to offer a way out: the model has to answer to something outside itself. Yet the same problem appears at the bench. A measurement can be real and still reward the wrong thing.

Berkeley’s A-Lab already runs materials experiments overnight, though some failures still need a person on-site. ¹ Its work also shows why running the experiment is only part of the problem. A later correction left four reported material identifications inconclusive and excluded a target already in the training data. ¹⁹

The lab made samples. Establishing what those samples were took another kind of work. A robot can follow the procedure correctly while the system draws the wrong conclusion from the result.

The same gap appears in biology, where a test often measures something easier to observe than the thing we actually care about.

A test for living cells, for example, can use light as a stand-in for cell count. But a compound can block luciferase, the enzyme that produces the light. The NIH’s Assay Guidance Manual describes this interference and recommends controls and other ways to measure the effect. ²¹

Less light may mean the compound blocked the reporter, not that it killed cells. A model rewarded for reducing the light can get a better score by finding compounds that interfere with the test.

Running that assay faster would produce more of the same uncertainty. The useful next experiment is one that separates the explanations: test whether the compound affects the reporter without cells, or measure the cells another way. Those tests matter because their answers change what the lab should believe and do next.

This is where execution becomes investigation. The system has to question what its own measurement means.

Even when an effect is real, the explanation can be wrong.

Jason Sheltzer’s lab tested ten cancer compounds. The drugs still had anticancer effects in the tested cells after the researchers removed their proposed targets. The effect survived; the proposed mechanism did not explain it. ²²

That experiment asks more than whether a drug works. If its effect requires a particular protein, removing that protein should change the result. The test needs care: the researchers must check that the gene was disabled and consider whether the cells found another way to do its job. But the proposed explanation now has something specific to answer for.

This is the value of a mechanism. It helps identify an experiment whose result could force a revision. Repeating a promising assay may make the estimate more precise while leaving the explanation just as uncertain.

For an autonomous lab, the hard choice is often which uncertainty to resolve. Another measurement of the same effect, a test of the proposed target, and a test in healthy cells answer different questions. The right choice depends on what is already known, what the research is for, and what each answer would make worth doing next.

That choice also matters because experiments are slow. Cells must grow and reactions must finish. A model that proposes a hundred experiments in seconds has not saved much time if none addresses the uncertainty holding up the investigation. More useful decisions can matter as much as faster equipment.

This changes what counts as useful training data. A successful result alone leaves out much of the work: why the experiment was chosen, what was expected, what failed, and why the next test was different.

I am building in the market for these experiments and their data. The distinction matters there too. A record that says “complete” is easy to sell and hard to learn from if it hides a failed transfer. Buyers need enough detail to tell whether the experiment happened as described and whether it answered the question they paid for.

The same is true of scientific mistakes. A record that preserves the reporter interference and the control result can teach something that the final cell-count measurement alone cannot. It shows how an apparently promising result was questioned.

There are two related abilities to learn here. One is predicting what an experiment will show. The other is choosing which experiment will be worth running. A model can improve at the first without improving at the second.

More data is useful when it improves those abilities beyond the experiments it came from. For predictions, that means testing new interventions, cell types and lab conditions, with simpler methods as comparisons. For experiment choice, it means checking whether the model reaches a sound conclusion with less time or fewer samples. A higher score on one assay does not establish either.

The physical work still has to hold up. Missing tips, warped plates and failed transfers can turn a sound plan into a misleading record. Detecting and recovering from those failures belongs in the learning loop. So does recognizing when a run cannot support a conclusion at all.

The promise of training in the real world is therefore conditional. Reality supplies feedback, but the lab must learn to read it.

In the 1940s, Barbara McClintock followed changes in corn-kernel colors across generations and connected them to changes in chromosomes. Some genetic elements could move within the genome. Evelyn Fox Keller called her biography “A Feeling for the Organism.”

What matters to me in that phrase is the attention behind it. Knowing an organism well means noticing when it does something your explanation cannot account for. The surprise becomes a reason to look more closely.

A machine would not need to share that feeling to do useful science. It would need to do the work the feeling describes: keep track of what it expected, notice the mismatch, and choose a test that helps explain it. That is what I mean by a mechanistic fallibilist. It has an account of how something works, and it can use evidence to find where that account breaks.

The earlier examples show why this matters. A material label can be wrong. A reporter can interfere with a measurement. A drug can have an effect for a different reason than expected. Each failure calls for a different next experiment. Treating them all as low scores would lose the lesson.

The CRO business could help build the machinery for this kind of learning. Paid experiments support equipment, reliable procedures and detailed records. They also expose systems to the awkward details that disappear from clean datasets.

But the business can succeed without solving the whole scientific problem. A service that reliably runs the tests its customers request is valuable even if the customers supply all the judgment about what to do next. There is no guarantee that more automated execution will turn into autonomous investigation.

I see closing that gap as the ambition for the next decade. The evidence of progress will be in the decisions: whether a lab catches a misleading result, abandons a failed explanation, or chooses a control that saves weeks of work. Those are stronger signs than the number of experiments it can run unattended.

People would still set the goals and remain responsible for what happens with the answers. Patients and clinicians, for example, live with the trade-offs between a stronger drug and a gentler one. A lab can help compare those options without deciding what people should value.

The next step is a lab that can take a goal and keep making useful decisions when the results stop fitting the plan. That is when it starts to carry an investigation forward.