Reaction kinetics
Predict reaction curves from a few measurements.
Take three measurements, then predict concentrations at the listed times.
3 / 3budget remaining
Experiments
Initial observations
[
{
"query": {
"time": 0.5,
"species": "A",
"concentration": 1
},
"values": [
0.7388413665274357
],
"noise_std": [
0.02
]
}
]Predictions
Model and scoring details
{
"task_id": "reaction-kinetics",
"version": "0.2.0",
"budget": 3,
"cost_semantics": "Every accepted experiment costs one credit; at most two repeats per action. Invalid requests cost nothing. The budget is a hard constraint; unused credits do not directly increase the score.",
"measurement_noise": "Independent additive zero-mean Gaussian noise per output and replicate, with the listed fixed standard deviations. Readouts are not clipped. Each measurement has equal synthetic exposure/cost. Repeating observe_initial returns the same initial data, not new measurements.",
"split_definition": "Train and test use independent parameter/noise draws from the same published distributions. No claim of out-of-family generalization.",
"status": "Original synthetic prototype; no physical calibration or LLM training result.",
"submission": "Submit one finite numeric matrix in the exact prediction_shape. Submission is terminal and cannot be retried after acceptance.",
"scoring": {
"name": "inverse_normalized_mse",
"formula": "score = 1 / (1 + mean(((prediction - noiseless_hidden_target) / scale)^2))",
"scale": [
0.02
],
"range": [
0,
1
],
"meaning": "Higher is better. Exact latent-mean prediction scores 1. A normalized RMSE equal to one measurement-noise standard deviation scores 0.5. Measurement noise is not added to hidden targets."
},
"title": "Reaction mechanism identification",
"outputs": [
"concentration"
],
"noise_std": [
0.02
],
"instructions": "Distinguish two idealized mass-conserving reaction mechanisms by choosing time/species assays under a three-credit budget. Predict the listed unmeasured early, middle and late time-course concentrations. Initial concentration and mechanism family are known; mechanism and rates are hidden.",
"model": "Either sequential A -> B -> C with rates k1,k2, or parallel A -> B and A -> C with rates k1,k2. First-order irreversible kinetics, initial A=c and B=C=0; no physical calibration.",
"prior": "Mechanism is equally likely. Rates k1,k2 independently log-Uniform(0.1,0.75), in inverse synthetic time units.",
"initial_queries": [
{
"time": 0.5,
"species": "A",
"concentration": 1
}
],
"prediction_queries": [
{
"time": 0.4,
"species": "A",
"concentration": 1
},
{
"time": 0.4,
"species": "B",
"concentration": 1
},
{
"time": 0.4,
"species": "C",
"concentration": 1
},
{
"time": 1.25,
"species": "A",
"concentration": 0.7
},
{
"time": 1.25,
"species": "B",
"concentration": 0.7
},
{
"time": 1.25,
"species": "C",
"concentration": 0.7
},
{
"time": 3.5,
"species": "A",
"concentration": 1
},
{
"time": 3.5,
"species": "B",
"concentration": 1
},
{
"time": 3.5,
"species": "C",
"concentration": 1
},
{
"time": 7.5,
"species": "A",
"concentration": 1.3
},
{
"time": 7.5,
"species": "B",
"concentration": 1.3
},
{
"time": 7.5,
"species": "C",
"concentration": 1.3
},
{
"time": 10,
"species": "A",
"concentration": 1
},
{
"time": 10,
"species": "B",
"concentration": 1
},
{
"time": 10,
"species": "C",
"concentration": 1
}
],
"experiments": [
{
"id": "assay-A-0",
"label": "Assay A at t=0.25",
"query": {
"time": 0.25,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-0",
"label": "Assay B at t=0.25",
"query": {
"time": 0.25,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-0",
"label": "Assay C at t=0.25",
"query": {
"time": 0.25,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-1",
"label": "Assay A at t=0.75",
"query": {
"time": 0.75,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-1",
"label": "Assay B at t=0.75",
"query": {
"time": 0.75,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-1",
"label": "Assay C at t=0.75",
"query": {
"time": 0.75,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-2",
"label": "Assay A at t=1.5",
"query": {
"time": 1.5,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-2",
"label": "Assay B at t=1.5",
"query": {
"time": 1.5,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-2",
"label": "Assay C at t=1.5",
"query": {
"time": 1.5,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-3",
"label": "Assay A at t=2.5",
"query": {
"time": 2.5,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-3",
"label": "Assay B at t=2.5",
"query": {
"time": 2.5,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-3",
"label": "Assay C at t=2.5",
"query": {
"time": 2.5,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-4",
"label": "Assay A at t=4",
"query": {
"time": 4,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-4",
"label": "Assay B at t=4",
"query": {
"time": 4,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-4",
"label": "Assay C at t=4",
"query": {
"time": 4,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-5",
"label": "Assay A at t=6",
"query": {
"time": 6,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-5",
"label": "Assay B at t=6",
"query": {
"time": 6,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-5",
"label": "Assay C at t=6",
"query": {
"time": 6,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-6",
"label": "Assay A at t=8",
"query": {
"time": 8,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-6",
"label": "Assay B at t=8",
"query": {
"time": 8,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-6",
"label": "Assay C at t=8",
"query": {
"time": 8,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
}
],
"action_schema": {
"type": "object",
"properties": {
"action_id": {
"type": "string",
"enum": [
"assay-A-0",
"assay-B-0",
"assay-C-0",
"assay-A-1",
"assay-B-1",
"assay-C-1",
"assay-A-2",
"assay-B-2",
"assay-C-2",
"assay-A-3",
"assay-B-3",
"assay-C-3",
"assay-A-4",
"assay-B-4",
"assay-C-4",
"assay-A-5",
"assay-B-5",
"assay-C-5",
"assay-A-6",
"assay-B-6",
"assay-C-6"
]
}
},
"required": [
"action_id"
],
"additionalProperties": false
},
"prediction_shape": [
15,
1
]
}Synthetic example with public answers. This is not a secure benchmark.
Task details
Distinguish two idealized mass-conserving reaction mechanisms by choosing time/species assays under a three-credit budget.
Sequential A → B → C and parallel A → B / A → C mechanisms can explain limited initial measurements differently. Choose time and species assays that help predict the unseen concentration curves.
Actions
- Measure. Select a permitted experimental condition and collect a noisy readout.
- Predict. Commit to predictions for the held-out conditions.
Sample run
Initial observations
The measurements available before the policy spends its experimental budget.
[
{
"query": {
"time": 0.5,
"species": "A",
"concentration": 1
},
"values": [
0.7708817736901392
],
"noise_std": [
0.02
]
}
]Upper median score among all completed adaptive-classical runs, selected for illustration after evaluation; not an LLM rollout or a best run. Estimator: Finite-grid posterior over both mechanisms and 41x41 log-spaced rate pairs
Read the full task specification
{
"task_id": "reaction-kinetics",
"version": "0.2.0",
"budget": 3,
"cost_semantics": "Every accepted experiment costs one credit; at most two repeats per action. Invalid requests cost nothing. The budget is a hard constraint; unused credits do not directly increase the score.",
"measurement_noise": "Independent additive zero-mean Gaussian noise per output and replicate, with the listed fixed standard deviations. Readouts are not clipped. Each measurement has equal synthetic exposure/cost. Repeating observe_initial returns the same initial data, not new measurements.",
"split_definition": "Train and test use independent parameter/noise draws from the same published distributions. No claim of out-of-family generalization.",
"status": "Original synthetic prototype; no physical calibration or LLM training result.",
"submission": "Submit one finite numeric matrix in the exact prediction_shape. Submission is terminal and cannot be retried after acceptance.",
"scoring": {
"name": "inverse_normalized_mse",
"formula": "score = 1 / (1 + mean(((prediction - noiseless_hidden_target) / scale)^2))",
"scale": [
0.02
],
"range": [
0,
1
],
"meaning": "Higher is better. Exact latent-mean prediction scores 1. A normalized RMSE equal to one measurement-noise standard deviation scores 0.5. Measurement noise is not added to hidden targets."
},
"title": "Reaction mechanism identification",
"outputs": [
"concentration"
],
"noise_std": [
0.02
],
"instructions": "Distinguish two idealized mass-conserving reaction mechanisms by choosing time/species assays under a three-credit budget. Predict the listed unmeasured early, middle and late time-course concentrations. Initial concentration and mechanism family are known; mechanism and rates are hidden.",
"model": "Either sequential A -> B -> C with rates k1,k2, or parallel A -> B and A -> C with rates k1,k2. First-order irreversible kinetics, initial A=c and B=C=0; no physical calibration.",
"prior": "Mechanism is equally likely. Rates k1,k2 independently log-Uniform(0.1,0.75), in inverse synthetic time units.",
"initial_queries": [
{
"time": 0.5,
"species": "A",
"concentration": 1
}
],
"prediction_queries": [
{
"time": 0.4,
"species": "A",
"concentration": 1
},
{
"time": 0.4,
"species": "B",
"concentration": 1
},
{
"time": 0.4,
"species": "C",
"concentration": 1
},
{
"time": 1.25,
"species": "A",
"concentration": 0.7
},
{
"time": 1.25,
"species": "B",
"concentration": 0.7
},
{
"time": 1.25,
"species": "C",
"concentration": 0.7
},
{
"time": 3.5,
"species": "A",
"concentration": 1
},
{
"time": 3.5,
"species": "B",
"concentration": 1
},
{
"time": 3.5,
"species": "C",
"concentration": 1
},
{
"time": 7.5,
"species": "A",
"concentration": 1.3
},
{
"time": 7.5,
"species": "B",
"concentration": 1.3
},
{
"time": 7.5,
"species": "C",
"concentration": 1.3
},
{
"time": 10,
"species": "A",
"concentration": 1
},
{
"time": 10,
"species": "B",
"concentration": 1
},
{
"time": 10,
"species": "C",
"concentration": 1
}
],
"experiments": [
{
"id": "assay-A-0",
"label": "Assay A at t=0.25",
"query": {
"time": 0.25,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-0",
"label": "Assay B at t=0.25",
"query": {
"time": 0.25,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-0",
"label": "Assay C at t=0.25",
"query": {
"time": 0.25,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-1",
"label": "Assay A at t=0.75",
"query": {
"time": 0.75,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-1",
"label": "Assay B at t=0.75",
"query": {
"time": 0.75,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-1",
"label": "Assay C at t=0.75",
"query": {
"time": 0.75,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-2",
"label": "Assay A at t=1.5",
"query": {
"time": 1.5,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-2",
"label": "Assay B at t=1.5",
"query": {
"time": 1.5,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-2",
"label": "Assay C at t=1.5",
"query": {
"time": 1.5,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-3",
"label": "Assay A at t=2.5",
"query": {
"time": 2.5,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-3",
"label": "Assay B at t=2.5",
"query": {
"time": 2.5,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-3",
"label": "Assay C at t=2.5",
"query": {
"time": 2.5,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-4",
"label": "Assay A at t=4",
"query": {
"time": 4,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-4",
"label": "Assay B at t=4",
"query": {
"time": 4,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-4",
"label": "Assay C at t=4",
"query": {
"time": 4,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-5",
"label": "Assay A at t=6",
"query": {
"time": 6,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-5",
"label": "Assay B at t=6",
"query": {
"time": 6,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-5",
"label": "Assay C at t=6",
"query": {
"time": 6,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-A-6",
"label": "Assay A at t=8",
"query": {
"time": 8,
"species": "A",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-B-6",
"label": "Assay B at t=8",
"query": {
"time": 8,
"species": "B",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
},
{
"id": "assay-C-6",
"label": "Assay C at t=8",
"query": {
"time": 8,
"species": "C",
"concentration": 1
},
"cost": 1,
"max_replicates": 2
}
],
"action_schema": {
"type": "object",
"properties": {
"action_id": {
"type": "string",
"enum": [
"assay-A-0",
"assay-B-0",
"assay-C-0",
"assay-A-1",
"assay-B-1",
"assay-C-1",
"assay-A-2",
"assay-B-2",
"assay-C-2",
"assay-A-3",
"assay-B-3",
"assay-C-3",
"assay-A-4",
"assay-B-4",
"assay-C-4",
"assay-A-5",
"assay-B-5",
"assay-C-5",
"assay-A-6",
"assay-B-6",
"assay-C-6"
]
}
},
"required": [
"action_id"
],
"additionalProperties": false
},
"prediction_shape": [
15,
1
]
}Evaluation
Reward is 1 / (1 + normalized MSE) against noiseless target means. Higher is better. Each output error is normalized by its published scale (0.02). Experimental cost is enforced through the budget.
| Policy | Episodes | Mean reward | 95% bootstrap interval | Mean RMSE |
|---|---|---|---|---|
| No experiments | 30 | 0.0458 | 0.0263–0.0697 | 0.1458 |
| Random experiments | 30 | 0.4950 | 0.3727–0.6119 | 0.0459 |
| Classical experimental design | 30 | 0.7816 | 0.7122–0.8466 | 0.0104 |
Paired worlds; random and classical design use identical published budgets (circuits8, kinetics3, spectra8) and the same estimator within each family. Spectra also includes a prior-fixed design diagnostic with the same estimator. Identical action/replicate pairs have identical noise. No-experiment uses only free initial observations. 95% percentile bootstrap CIs resample worlds 4000 times. Seed base and split are explicit and fixed before evaluation. Independent worlds use the same published prior. These numerical policies do not establish RL training gain. One in-session agent trial on a separate fresh world scored 0.5045 (RMSE 0.01982, 3 experiments). This is one protocol-blinded attempt, not a model benchmark; its complete trace is downloadable.
Sources and limitations
- Identifying Bayesian optimal experiments for uncertain biochemical pathway modelsNatalie M. Isenberg, Susan D. Mertins, Byung-Jun Yoon, Kristofer G. Reyes, Nathan M. Urban · 2024
Measure only useful species, prediction-focused design and explicit simulation-to-reality boundary.
The application is pharmacodynamic and uses model assumptions plus synthetic measurements. We borrow the general design formulation only; no therapeutic task or claim is made.
- Discrimination Among Mechanistic ModelsGeorge E. P. Box, William J. Hill · 1967
Foundational candidate-disagreement acquisition for the kinetic demo.
Assumes candidate model structures and tractable uncertainty approximations; identifying one candidate does not establish that nature belongs to the candidate set.
- Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihoodAndreas Raue, Clemens Kreutz, Tim Maiwald, Jasmin Bachmann, Marcel Schilling, Ursula Klingmüller, Jens Timmer · 2009
Reject impossible exact-parameter graders; prefer identifiable predictions or equivalence classes.
Profile likelihood itself is not a universal identifiability proof; every new task needs its own separation and sensitivity checks.