A task changes the scientific decision an agent must make. An instance changes the hidden system within that task.
50 tasks · 6 families · 3 implemented
This tree is an expansion plan. Only tasks marked Implemented have a corresponding environment in this release. Proposed tasks still require implementation, evaluation, and review.
Changing a random seed does not create a new task. The proposed distinctions concern mechanisms, intervention choices, measurement constraints, or the inference required.
50 tasks
causal-circuits / 10 tasks
Causal intervention circuits
Identify which interventions distinguish competing causal explanations, then predict unseen manipulations of small synthetic circuits.
A known nonlinear causal circuit has seven hidden coefficients; choose single-variable interventions to predict unseen compound interventions.
Task design & sources
Experimental budget
8 measurement credits. Per-action repetition cap is two total executions; each execution costs one credit. Invalid actions cost nothing; free initial observations are fixed and cannot be remeasured for free.
Experiment
Set a permitted input or clamp the internal x node; execute each action at most twice within the total budget.
Observation
Noisy x and y readouts from x=tanh(bx+wu*u+wv*v), y=tanh(by+wx*x+yu*u+yv*v). The graph and function class are public; coefficients are hidden.
Submission
Predict x and y for eight public compound-intervention queries; their correct values remain private. No graph submission is required.
Reward
Score=1/(1+NMSE), normalized by measurement-noise SD 0.035. Raw RMSE is reported. Two of 16 target entries are x values explicitly clamped in public target queries, so the complete-state score includes two directly known entries.
Held-out challenge
Independent coefficient/noise draws from the same public prior; compound queries are disjoint from initial and purchasable measurements. No unknown-graph or out-of-family evaluation.
Why this is distinct
Tests identification of causal response coefficients using interventions under a known graph, with transfer from single to compound perturbations. It does not test discovery of an unknown graph.
Baseline policies
Free-initial-observation estimator (no purchased experiments); Random intervention acquisition + shared estimator; Approximate uncertainty-directed acquisition + shared estimator
Acceptance gate
Core implementation and mathematical invariants locally checked. Buyer acceptance, target-agent difficulty, RL training utility and physical validation remain outstanding.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A convergent collider and a transmission chain can fit incomplete initial records but respond differently when an internal node is fixed.
Task design & sources
Experimental budget
18 combined intervention and readout credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Select an intervention target and value; optionally purchase an additional node readout.
Observation
Partial noisy node measurements with explicit observation-selection rules.
Submission
Predict intervention effects on downstream nodes and choose between the candidate structural classes.
Reward
Held-out response error plus separate class Brier score; do not score unobservable edges.
Held-out challenge
Disjoint graph realizations with matched observation marginals and novel intervention values.
Why this is distinct
Introduces collider structure and a readout-selection decision, changing which observation is informative.
Baseline policies
Fixed endpoint probes; Random target/readout pairs; Information-gain candidate planner
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
An output follows AND-like, OR-like or additive integration of two inputs, with thresholds hidden.
Task design & sources
Experimental budget
20 perturbation credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Set both inputs singly or jointly and choose a measurement dose.
Observation
Noisy bounded output response; controls reveal baseline and maximum response.
Submission
Predict a held-out two-input response surface and interaction sign.
Reward
Mean prediction error across the held-out input grid; interaction classification reported separately.
Held-out challenge
Novel threshold ranges and held-out input combinations, not merely fresh observation noise.
Why this is distinct
Single-input experiments may be insufficient; the task requires combinatorial intervention design.
Baseline policies
One-factor-at-a-time design; Full factorial coarse grid; Adaptive model-disagreement design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Two paths have opposing effects, so a near-zero total response may hide large causal influences.
Task design & sources
Experimental budget
20 perturbation credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Activate an upstream input and independently block each candidate intermediate pathway.
Observation
Noisy endpoint and selected intermediate readouts.
Submission
Predict double-intervention responses and explain whether the zero response reflects absence or cancellation.
Reward
Held-out double-intervention error; mechanism-class posterior as diagnostic.
Held-out challenge
New balanced and unbalanced pathway strengths; label permutations; double perturbations withheld.
Why this is distinct
Disentangles causal cancellation from no connection; requires interventions on intermediates rather than more replicates of one endpoint.
Baseline policies
Total-effect-only heuristic; Balanced pathway blocking; Bayesian model discrimination
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
An incoherent feed-forward mechanism creates a transient pulse after sustained input, unlike a monotone cascade.
Task design & sources
Experimental budget
24 measurement credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Apply step, pulse or removal stimuli; choose early and late sampling times.
Observation
Noisy output trajectories with optional sparse intermediate measurements.
Submission
Predict peak time, peak height and late response for novel stimuli.
Reward
Held-out full-trajectory loss; peak metrics are secondary diagnostics to prevent peak-only gaming.
Held-out challenge
Novel pulse schedules and rate ratios with sufficient output resolution.
Why this is distinct
Needs transient shape and signed-pathway reasoning; differs from the duration-threshold objective in causal-06.
Baseline policies
Endpoint-only design; Uniform time sampling; Bayesian transient-disagreement design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Development contexts share a mechanism but differ in input distribution; a deployment context may expose one changed mechanism.
Task design & sources
Experimental budget
24 experiment credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Allocate interventions to two development contexts and buy a limited diagnostic in the deployment context.
Observation
Noisy context-labeled responses and covariates.
Submission
Predict deployment interventions and identify the claimed invariant mechanism subset.
Reward
Deployment prediction error, with separate diagnostics for false invariance and calibration.
Held-out challenge
Entire deployment regimes and changed mechanisms withheld at training time.
Why this is distinct
Requires separating reusable structure from context-specific change; independent seeds of one fixed distribution do not supply this challenge.
Baseline policies
Pool all contexts; Fit each context independently; Invariant-plus-change-point model
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A→B→C and parallel A→B/A→C mechanisms share reactant disappearance patterns but differ in species time courses.
Task design & sources
Experimental budget
3 species-assay credits. Per-action repetition cap is two total executions; each execution costs one credit. Invalid actions cost nothing; free initial observations are fixed and cannot be remeasured for free.
Experiment
Choose one of 21 public species/time assays at times 0.25, 0.75, 1.5, 2.5, 4, 6 or 8. Each assay may be executed at most twice; every execution costs one of three credits.
Observation
Noisy concentration; two hidden first-order rate constants sampled in [0.1,0.75].
Submission
Predict A/B/C concentrations for public target (time,initial concentration) pairs (0.4,1.0), (1.25,0.7), (3.5,1.0), (7.5,1.3), (10,1.0); target values remain private.
Reward
Score=1/(1+NMSE), normalized by assay-noise SD 0.02; raw RMSE reported. Target panel spans early/middle/late dynamics rather than late endpoints only.
Held-out challenge
New rate/noise draws from the same public prior and disjoint time/concentration queries. Both mechanism classes remain available in both splits. Known linear scaling with initial concentration is not novel mechanism transfer.
Why this is distinct
Requires identifying pathway structure through time-dependent species measurements; reactant disappearance alone does not fully separate the mechanisms.
Baseline policies
Free-initial-observation finite-grid posterior; Random species/time acquisition + shared posterior; Target-variance acquisition + same 41x41 grid over both mechanisms
Acceptance gate
Core implementation and mathematical invariants locally checked. Buyer acceptance, target-agent difficulty, RL training utility and physical validation remain outstanding.
A reaction is approximately linear at low substrate levels but may saturate outside that regime.
Task design & sources
Experimental budget
18 rate-measurement credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose substrate concentration and replication count; initial-rate measurements only.
Observation
Noisy initial rates with known measurement variance model.
Submission
Predict an unseen concentration-response curve and estimate the saturation regime.
Reward
Held-out curve error weighted equally across low and high regimes.
Held-out challenge
Hold out the transition-to-saturation interval and new capacity/affinity pairs.
Why this is distinct
Requires deliberately leaving the locally linear regime; more low-range repeats cannot identify saturation.
Baseline policies
Low-concentration linear fit; Log-spaced concentration grid; D-optimal or disagreement design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A plateau may indicate reversible equilibrium or irreversible depletion of a limiting reactant.
Task design & sources
Experimental budget
24 sampling and perturbation credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose starting amounts, spike one species after a plateau, and sample over time.
Observation
Noisy concentrations of declared measurable species; the spike is recorded in the mass balance.
Submission
Predict response to a new post-plateau spike and final composition.
Reward
Held-out dynamic response error and conservation diagnostic.
Held-out challenge
New initial ratios and spike schedules; withheld species-spike combinations.
Why this is distinct
Changes the experimental logic from observing a plateau to actively testing reversibility.
Baseline policies
Fit plateau height only; Fixed reversal/spike protocol; Adaptive mechanistic discrimination
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Entire initial-concentration levels and longer times withheld; both model classes represented.
Why this is distinct
Changes reaction order rather than network topology; concentration scaling, not species selection, is the key discriminating intervention.
Baseline policies
Fit one decay curve; Concentration-by-time grid; Candidate-disagreement design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
New deactivation and inhibition strengths; withheld recovery sequences.
Why this is distinct
Tests recovery interventions on a stateful process; passive deceleration curves are deliberately ambiguous.
Baseline policies
Fit a declining-rate curve; Fixed fresh-catalyst/product-removal controls; Bayesian mechanistic discrimination
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
An apparent rate law hides two pathways whose relative contributions shift with temperature.
Task design & sources
Experimental budget
24 condition-assay credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose a dimensionless temperature setting and substrate level, then assay product composition.
Observation
Noisy total rate and product selectivity in an explicitly synthetic safe-range model.
Submission
Predict both rate and composition at unseen temperature conditions.
Reward
Joint held-out rate/selectivity error; normalization fixed from training reference ranges.
Held-out challenge
Unseen temperature intervals and activation-scale contrasts, within declared simulator bounds.
Why this is distinct
Requires composition measurements to detect competing pathways; a good total-rate fit can be misleading.
Baseline policies
Single Arrhenius-like fit; Balanced temperature ladder; Multi-output model-disagreement planner
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A steep aggregate response may reflect cooperative activation or a mixture of independent subpopulations.
Task design & sources
Experimental budget
24 assay credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose concentration and independently fractionate a declared subpopulation or apply a selective probe.
Observation
Noisy total and optional fraction-specific rates.
Submission
Predict response after a new fractionation/probe condition and calibrate mechanism probability.
Reward
Held-out intervention prediction error; class score only on separable instances.
Held-out challenge
New mixture proportions and affinity contrasts; exclude observationally indistinguishable parameter regions.
Why this is distinct
Introduces population heterogeneity and a selective measurement; steepness alone must not be treated as proof of cooperativity.
Baseline policies
Single Hill-curve fit; Fixed fractionation panel; Adaptive competing-model design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Nine candidate spectral libraries differ in global peak shift and broadening; hidden mixture amplitudes and affine background further obscure unmeasured wavelengths.
Task design & sources
Experimental budget
8 wavelength-measurement credits. Per-action repetition cap is two total executions; each execution costs one credit. Invalid actions cost nothing; free initial observations are fixed and cannot be remeasured for free.
Experiment
Choose one of 49 wavelength measurements (405 through 645 in steps of 5); each may be executed at most twice within eight credits.
Observation
Additive Gaussian signal noise SD 0.025. Base Gaussian peaks centered at 447,482,531 with widths10,18,12; one hidden global shift in {-8,0,8} and width multiplier in {0.75,1,1.25}; unknown three amplitudes, offset and slope. Units are synthetic; wavelength coordinate uses nm.
Submission
Predict signal for 12 public unmeasured wavelengths; correct values are private.
Reward
Score=1/(1+NMSE), normalized by noise SD 0.025; raw RMSE reported. No photon-counting or physical-identity score.
Held-out challenge
Independent library, coefficient and noise draws from the same public prior; target wavelengths disjoint from initial and purchased queries. No new-library or chemical-domain generalization is established.
Why this is distinct
Joint spectral-library discrimination and mixture/background inference from selective measurements. The model probabilities depend on observed values, unlike fixed single-library linear-Gaussian covariance design.
Baseline policies
Free-initial-observation spectrum estimator; Random wavelength acquisition + same estimator; Readout-dependent nine-library posterior / target-variance acquisition + same estimator
Acceptance gate
Core implementation and mathematical invariants locally checked. Buyer acceptance, target-agent difficulty, RL training utility and physical validation remain outstanding.
A single broad peak and two close narrow peaks can agree at a few initial wavelengths.
Task design & sources
Experimental budget
60 exposure-time units. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Select wavelength and exposure duration from a fixed total exposure budget.
Observation
Poisson counts plus known detector background; count rate depends on exposure.
Submission
Predict the full held-out spectrum and the probability of one versus two peaks.
Reward
Exposure-matched spectral predictive error and class Brier score.
Held-out challenge
New peak separations, widths and count rates; include difficult but identifiable regimes.
Why this is distinct
Peak count is unknown, so parameter estimation within one fixed model is insufficient.
Baseline policies
Uniform scan; Random wavelengths; Bayesian model-selection acquisition
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A broad spectral feature and a smooth baseline can explain the same local observations.
Task design & sources
Experimental budget
24 sample/reference measurement credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose wide-range wavelengths and optional blank-reference measurements.
Observation
Noisy sample and blank signals, with explicit shared and sample-specific background terms.
Submission
Predict baseline-corrected signal and a held-out spectrum under a new baseline offset.
New background curvature and broad-feature width; controls remain informative.
Why this is distinct
Needs reference measurements and coverage outside the apparent feature rather than simply denser local sampling.
Baseline policies
Local peak-only scan; Wide uniform scan plus blanks; Joint background/model-disagreement design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Signal may be a linear combination of components or become nonlinear with concentration/path length.
Task design & sources
Experimental budget
24 aliquot-measurement credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose dilution, optical path setting and wavelength on fresh aliquots.
Observation
Noisy signal and known dilution/path metadata.
Submission
Predict a held-out concentration-by-path spectral panel.
Reward
Balanced held-out spectral prediction loss across linear and nonlinear regimes.
Held-out challenge
New high-concentration regimes and path settings; synthetic model class disclosed.
Why this is distinct
Uses controlled path/dilution interventions to test the measurement law; coefficients alone cannot capture the change.
Baseline policies
Linear unmixing only; Dilution/path factorial; Competing optical-model design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A weak component is identifiable only when measurement time is allocated between informative wavelengths and background estimation.
Task design & sources
Experimental budget
80 total dwell-time units. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose wavelength and dwell time; exposure is charged continuously.
Observation
Poisson photon counts with an unknown nuisance background.
Submission
Predict the weak-component contribution and held-out spectrum with uncertainty.
Reward
Spectral predictive loss at matched total exposure; weak-component detection Brier score separate.
Held-out challenge
New intensity/background ratios and nonuniform detector efficiency.
Why this is distinct
Changes the design resource from number of queries to exposure allocation; repeated short queries are not treated as free new tasks.
Baseline policies
Equal exposure per wavelength; Peak-only exposure; Expected-information-per-time design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Exposure irreversibly bleaches one component, so late observations no longer represent the initial sample.
Task design & sources
Experimental budget
60 exposure units. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose wavelength, duration and whether to consume a fresh aliquot.
Observation
Noisy spectrum point with cumulative exposure history and finite aliquot inventory.
Submission
Reconstruct the initial spectrum and forecast the spectrum after a new exposure schedule.
Reward
Initial-spectrum and future-schedule prediction losses; aliquot use reported separately.
Held-out challenge
New bleaching rates and measurement orderings; hold out exposure sequences.
Why this is distinct
Measurement changes the latent state; order matters, unlike independent reset spectroscopy.
Baseline policies
Ignore bleaching; Short uniform exposures on fresh aliquots; Stateful Bayesian acquisition
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
New compositions and modality-specific nuisance parameters; one modality becomes weak in selected regimes.
Why this is distinct
Requires switching measurement modality to break a degeneracy, rather than adding more points from one sensor.
Baseline policies
One modality only; Fixed balanced modality split; Joint posterior-uncertainty-per-cost design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A sparse frequency mixture produces an interferogram; delay selection controls frequency-resolution ambiguity.
Task design & sources
Experimental budget
60 exposure units. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose a time delay and exposure duration, including revisits.
Observation
Noisy interferogram value or photon count with a disclosed forward transform.
Submission
Predict the held-out spectrum and interferogram at unseen delays.
Reward
Joint held-out reconstruction loss with fixed spectrum normalization.
Held-out challenge
New frequency separations and phases; delays chosen to expose aliasing rather than repeat one grid.
Why this is distinct
Actions are time-domain measurements; aliasing and phase ambiguity differ from direct wavelength sampling.
Baseline policies
Uniform delay sampling; Random delays; Information-optimal delay selection
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Unseen waveform classes and delay/time-constant combinations.
Why this is distinct
Requires time-structured excitation rather than static input-output regression.
Baseline policies
Single step sampled late; Fixed multiscale pulse train; Active input-excitation design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A second-order system may be underdamped or overdamped despite similar early displacement.
Task design & sources
Experimental budget
30 sensor/input credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose impulse size and sampling cadence under bounded state limits.
Observation
Noisy position and optionally costly velocity measurements.
Submission
Predict unseen free-decay and forced-response trajectories.
Reward
Held-out trajectory error and damping-regime probability.
Held-out challenge
New damping ratios and natural frequencies; exclude indistinguishable critical neighborhoods if class scoring is used.
Why this is distinct
Tests oscillation/damping and sensor choice rather than a first-order delay.
Baseline policies
Uniform position samples; Frequency sweep; Adaptive input plus sensor selection
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
A linear oscillator and a weakly nonlinear restoring force share small-amplitude behavior but differ at larger excursions.
Task design & sources
Experimental budget
30 trajectory credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose safe dimensionless initial displacement or forcing amplitude and observation times.
Observation
Noisy motion trajectory with declared state and amplitude limits.
Submission
Forecast novel-amplitude oscillations and estimate frequency shift.
Reward
Held-out trajectory loss across amplitude regimes; invariant checks separate.
Held-out challenge
Large-amplitude regimes withheld while remaining inside model domain.
Why this is distinct
Needs amplitude excitation to identify nonlinear force terms; changing resonance frequency alone would be only a parameter variant.
Baseline policies
Linear oscillator fit; Fixed amplitude ladder; Sparse-library active identification
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
New exchange rates, hidden volumes and injection locations.
Why this is distinct
Requires location-specific excitation and partial observation to resolve hidden exchange.
Baseline policies
Aggregate-only exponential fit; Fixed pulse-chase; Active pulse and observation selection
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Apparent material loss may be true leakage or transfer into a poorly observed internal store.
Task design & sources
Experimental budget
30 sensor/intervention credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose input pulses, isolation interventions and sensor placement among declared locations.
Observation
Noisy local/aggregate amounts and logged external inputs.
Submission
Forecast recovery after isolation and account for total material.
Reward
Mass-balance-consistent held-out prediction loss; no exact hidden-state grading when unidentifiable.
Held-out challenge
New leak rates, storage timescales and sensor biases with controls.
Why this is distinct
Distinguishes conservation failure from partial observability, requiring an isolation or recovery test.
Baseline policies
Assume conservation; Measure endpoint totals; Active conservation-residual diagnosis
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
New concentrations spanning below-range, in-range and saturated regimes; new calibration parameters.
Why this is distinct
Requires recovering the measurable range before inference, rather than fitting a fully observed curve.
Baseline policies
Use undiluted measurement; Fixed dilution ladder; Sequential range-finding plus calibration
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Allocate calibration standards under unequal noise
Calibration slope/curvature and signal-dependent variance are unknown; standards at the extremes and repeats have different value.
Task design & sources
Experimental budget
24 calibration/sample assay credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose standard concentration or unknown-sample measurement and number of replicates.
Observation
Noisy signals with variance depending on signal level.
Submission
Predict unknown concentrations and provide intervals for a held-out sample panel.
Reward
Mean predictive loss and interval score; empirical coverage and width separate.
Held-out challenge
New response scales, noise functions and concentrations; complete calibration batches held out.
Why this is distinct
Design targets inverse concentration inference under heteroscedasticity, not simply forward curve fit.
Baseline policies
Unweighted fixed-grid calibration; Weighted nonlinear regression on a fixed design; Bayesian inverse-prediction design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Two substances contribute to one assay channel; a high signal may be target analyte or an interferent.
Task design & sources
Experimental budget
24 assay/control credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose an orthogonal assay, a selective depletion control or a known spike.
Observation
Noisy channel responses with declared cross-sensitivity models.
Submission
Predict target concentration and the response to a new selective control.
Reward
Target prediction error and control-response error; false target attribution diagnostic.
Held-out challenge
New target/interferent mixtures and cross-sensitivity factors, with sufficient permitted controls.
Why this is distinct
Tests chemical/measurement specificity, distinct from multiplicative matrix suppression.
Baseline policies
Single-channel calibration; Fixed orthogonal-assay panel; Cost-aware discriminatory control selection
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
New carryover decay constants and sample permutations; order sequences held out as units.
Why this is distinct
Actions alter contamination state; an independent-query assay model cannot express the task.
Baseline policies
Ignore order; Blank after every sample; Adaptive wash-and-confirm policy
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Allocate experiments across fidelities, contexts, objectives, resources and delayed feedback.
allocation-01
Explore or confirm a noisy winner
The highest observed response may be a noisy outlier rather than the best reproducible condition.
Task design & sources
Experimental budget
24 experiment credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose a new condition or replicate an existing one under one budget.
Observation
Noisy scalar outcomes with condition-dependent variance and explicit independent replicates.
Submission
Nominate one condition and predict its independent confirmation mean.
Reward
Regret against the best allowed expected outcome using fresh confirmation draws; report estimation error separately.
Held-out challenge
New response landscapes and heteroscedasticity; confirmation RNG independent from exploration.
Why this is distinct
Separates exploration from confirmation and blocks reward based only on lucky observed maxima.
Baseline policies
Best observed single draw; Uniform replicated design; Bayesian best-arm identification
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Transfer from accessible proxies to a scarce target
Several accessible systems share partial structure with a target that cannot be experimentally screened during the episode.
Task design & sources
Experimental budget
30 proxy experiment credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Allocate experiments across proxy systems and conditions; choose a final target condition.
Observation
Noisy proxy responses and public system descriptors; no target feedback.
Submission
Predict target response across held-out conditions and nominate a final choice.
Reward
Hidden target prediction error plus confirmation regret; do not score proxy success as target success.
Held-out challenge
Entire target systems and a known subset of transfer-breaking mechanisms withheld.
Why this is distinct
Adds an explicit deployment target with no direct feedback; better proxy optimization is not automatically better target prediction.
Baseline policies
Pool proxies indiscriminately; Nearest-proxy transfer; Hierarchical transfer-aware design
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Two objectives compete, so no single setting maximizes both; useful experiments reveal the feasible Pareto trade-off.
Task design & sources
Experimental budget
30 experiment credits. Initial design specification; final API limits and acceptance thresholds require implementation and validation.
Experiment
Choose experiment conditions and optional replicates of ambiguous non-dominated candidates.
Observation
Noisy paired outcomes such as response quality and resource use, in normalized synthetic units.
Submission
Submit a small set of candidate conditions and predicted objective pairs.
Reward
Hypervolume or fixed-preference regret on independent confirmation measurements; uncertainty and infeasibility separate.
Held-out challenge
New trade-off shapes and held-out preference vectors; reference point fixed before evaluation.
Why this is distinct
Final output is a set of defensible trade-offs rather than a single scalar optimum.
Baseline policies
Single weighted-sum optimization; Space-filling design; Multi-objective Bayesian acquisition
Acceptance gate
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.
Oracle/public-interface feasibility; equal-budget baseline comparison; independent test worlds; identifiability and reward-leakage review; at least one demonstrated failure mode that changes the experimental policy.