Numerical and integration tests
53 core/bridge tests; Karotte task and permission checks; pinned supply-chain configuration.
Evidence and limitations
The literature motivates the task. Tests establish specific implementation behavior. Neither substitutes for validating a real scientific system.
Release 0.2.0
Implementation checks
Built with Karotte 3.0.72. Three end-to-end in-session runs, 53 numerical/bridge tests and 300 classical-policy runs completed. Docker integration and separate Apple VM confinement checks passed; mounted starts and VM artifact export remain unresolved on this host.
Classical baselines
300 completed runs on 30 new worlds per task, zero failures; separate kinetic shortcut and spectral fixed-design diagnostics.
In-session agent trials
One recorded attempt per task: causal 0.860, kinetic 0.504, spectral 0.870. Valid submissions, no post-score retries. These three attempts are not a broad model benchmark.
Submission custody
Missing, NaN, oversized, symlink and FIFO answers score zero; protected-data, shared-write and student-network probes blocked.
Scientific review
Primary-source mapping and a separate AI-assisted mathematical audit. No external human expert endorsement or physical calibration.
VM confinement check
Unmounted Apple VM run and Karotte confinement check passed: memory/process/file limits, PID namespace and firewall. Local mounted starts and VM artifact export stalled; the tested recipe and boundary are documented.
Primary sources
Sources
Each entry states the role of a source in the design. A related paper is not evidence that this prototype reproduces its results.
- dad2021
Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design ↗
Adam Foster, Desi R. Ivanova, Ilyas Malik, Tom Rainforth · 2021
Baseline and training formulation for sequential experiment selection; motivates comparing learned policies with online Bayesian design.
Scope: Demonstrated on the paper's model families; it does not prove LLM training gains or transfer to physical laboratories.
- tigas2023
Differentiable Multi-Target Causal Bayesian Experimental Design ↗
Panagiotis Tigas, Yashas Annadani, Desi R. Ivanova, Andrew Jesson, Yarin Gal, Adam Foster, Stefan Bauer · 2023
Joint interventions, batched experiment selection, strong non-LLM acquisition baseline.
Scope: Reported evaluations use synthetic datasets; multi-target design is not by itself a demonstration of realistic gene regulation.
- elahi2024
Adaptive Online Experimental Design for Causal Discovery ↗
Muhammad Qasim Elahi, Lai Wei, Murat Kocaoglu, Mahsa Ghasemi · 2024
Allocation, replicate-versus-new-intervention choices, stopping decisions and finite-sample comparisons.
Scope: Graph recovery metric and assumptions differ from predictive utility; its simulated-graph performance is not evidence for this new catalogue.
- zhang2023
Active learning for optimal intervention design in causal models ↗
Jiaqi Zhang, Louis Cammarata, Chandler Squires, Themistoklis P. Sapsis, Caroline Uhler · 2023
Distinguishes intervention optimization from graph discovery; target-relevant acquisition.
Scope: Known-graph linear theory must not be presented as general guarantees for nonlinear unknown graphs; biological application does not validate the proposed synthetic circuits.
- reisach2021
Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game ↗
Alexander G. Reisach, Christof Seiler, Sebastian Weichwald · 2021
Leakage audit, variance-only baseline, relabeling and rescaling stress tests.
Scope: Standardization alone does not guarantee removal of all generator artifacts; the paper does not certify any alternative generator.
- boxinggym2025
BoxingGym: Benchmarking Progress in Automated Experimental Design and Model Discovery ↗
Kanishk Gandhi, Michael Y. Li, Lyle Goodyear, Agam Bhatia, Louise Li, Aditi Bhaskar, Mohammed Zaman, Noah D. Goodman · 2025
Competitive context and direct baseline for the claimed product category.
Scope: The reviewed abstract reports models of its evaluation period; it does not establish performance of October 2026 models or demand for a commercial package.
- causalab2026
CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists ↗
Junlin Yang, Dylan Zhang, Xiangchen Song, Qirun Dai, Xiao Liu, Yuen Chen, Aniket Vashishtha, Jing Shi, Chenhao Tan, Hao Peng · 2026
Closest competitive prior art; separates prediction, structural fidelity and evidence-consistency diagnostics.
Scope: Uses synthetic linear/quadratic SCM families, not validated biological systems. Do not infer that a high prediction score establishes mechanism discovery.
- newtonbench2026
NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents ↗
Tianshi Zheng, Kelvin Kiu-Wai Tam, Newt Hue-Nam K. Nguyen, Baixuan Xu, Zhaowei Wang, Jiayang Cheng, Hong Ting Tsang, Weiqi Wang, Jiaxin Bai, Tianqing Fang, Yangqiu Song, Ginny Y. Wong, Simon See · 2026
Competitive context, shifted-mechanism splits and noise sensitivity.
Scope: Counterfactual laws improve novelty control but can reduce physical fidelity; catalogue size does not establish 324 independent simulator engines.
- scigym2025
Measuring Scientific Capabilities of Language Models with a Systems Biology Dry Lab ↗
Haonan Duan, Stephen Zhewen Lu, Caitlin Fiona Harrigan, Nishkrit Desai, Jiarui Lu, Michał Koziarski, Leonardo Cotta, Chris J. Maddison · 2025
Competitive context and path toward future curated biological models.
Scope: Simulation complexity and biological provenance do not prove physical transfer; these are existing systems, not assets licensed or incorporated here.
- nabika2024
Bayesian active learning with model selection for spectral experiments ↗
Tomohiro Nabika, Kenji Nagata, Masaichiro Mizumaki, Shun Katakami, Masato Okada · 2024
Primary mathematical anchor for adaptive peak discrimination, measurement-point choice and exposure budgets.
Scope: Results described include numerical XPS experiments; our Gaussian-mixture/linear-mixing demo is a simplified adaptation, not a replication or a chemically validated spectrometer.
- ito2025
Optimal spectroscopic measurement design: Bayesian framework for rational data acquisition ↗
Yusei Ito, Yasuo Takeichi, Hideitsu Hino, Kanta Ono · 2025
Known-library mixture identification and acquisition-budget reasoning.
Scope: The reported reduction in points is system-specific; spectral databases and calibration assumptions are not available automatically for this new environment.
- schroeder2025
Information-optimal measurement: From fixed sampling protocols to adaptive spectroscopy ↗
J. Schroeder, S. Howard, C. Eberle, J. Esslinger, N. Leopold-Kerschbaumer, K. V. Kepesidis, A. Döpp · 2025
Time-domain/Fourier sampling extension and distinction between measurement design and static reconstruction.
Scope: Strong guarantee language in the abstract is conditional on the paper's assumptions; no universal superiority claim is adopted. Title differs from earlier search indexing.
- isenberg2024
Identifying Bayesian optimal experiments for uncertain biochemical pathway models ↗
Natalie M. Isenberg, Susan D. Mertins, Byung-Jun Yoon, Kristofer G. Reyes, Nathan M. Urban · 2024
Measure only useful species, prediction-focused design and explicit simulation-to-reality boundary.
Scope: The application is pharmacodynamic and uses model assumptions plus synthetic measurements. We borrow the general design formulation only; no therapeutic task or claim is made.
- schorning2017
Optimal designs for enzyme inhibition kinetic models ↗
Kirsten Schorning, Holger Dette, Katrin Kettelhake, Tilman Möller · 2017
Two-factor enzyme-kinetic experiment design and D-optimal baseline.
Scope: Reviewed abstract specifically details noncompetitive-model parameter design; it does not validate all inhibition-discrimination scenarios in the catalogue.
- mangan2003
Structure and function of the feed-forward loop network motif ↗
Shmoolik Mangan, Uri Alon · 2003
Motif-based task distinctions that change necessary experiments, rather than relabeling one random graph.
Scope: Theoretical motif models are not a general GRN simulator; behavior depends on parameters and logic, so every generated task needs numerical validation.
- ma2009
Defining Network Topologies that Can Achieve Biochemical Adaptation ↗
Wenzhe Ma, Ala Trusina, Hana El-Samad, Wendell A. Lim, Chao Tang · 2009
Separate adaptation-mechanism tasks that need transient or targeted-intervention evidence.
Scope: The conclusion is scoped to the searched model class; not every feedback or feed-forward network adapts.
- fasel2022
Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control ↗
Urban Fasel, J. Nathan Kutz, Bingni W. Brunton, Steven L. Brunton · 2022
Strong explicit system-identification baseline and sparse-dynamics family.
Scope: Requires an appropriate candidate function library and data treatment; ensemble agreement does not guarantee true dynamics or scientific validity.
- xie2025
Online design of experiments by active learning for nonlinear system identification ↗
Kui Xie, Alberto Bemporad · 2025
Pulse design, stateful excitation, reset costs and control constraints.
Scope: Performance is reported on selected identification benchmarks; not a guarantee for novel dynamics or arbitrary LLM policies.
- helleckes2022
Bayesian calibration, process modeling and uncertainty quantification in biotechnology ↗
Laura Marie Helleckes, Michael Osthege, Wolfgang Wiechert, Eric von Lieres, Marco Oldiges · 2022
Calibration, saturation, heteroscedasticity and uncertainty propagation task family.
Scope: Source experiments concern particular biomass and glucose workflows; simplified synthetic assay models are not instrument-calibrated replacements.
- schoepfer2024
Cost-informed Bayesian reaction optimization ↗
Alexandre A. Schoepfer, Jan Weinreich, Ruben Laplaza, Jerome Waser, Clemence Corminboeuf · 2024
Inventory-aware and cost-aware experimental allocation; compare equal-currency rather than equal-query baselines.
Scope: Retrospective literature-data evaluation does not establish new physical experiments; reported savings are specific to tested datasets.
- angello2024
Closed-loop transfer enables artificial intelligence to yield chemical knowledge ↗
Nicholas H. Angello, David M. Friday, Changhyun Hwang, Seungjoo Yi, Austin H. Cheng, Tiara C. Torres-Flores, Edward R. Jira, Wesley Wang, Alán Aspuru-Guzik, Martin D. Burke, Charles M. Schroeder, Ying Diao, Nicholas E. Jackson · 2024
Distinguish best-observed optimization from transferable explanation; leave-one-context-out deployment tasks.
Scope: Real experimental validation belongs to the source authors; no such validation has been performed for our catalogue.
- volk2023
AlphaFlow: autonomous discovery and optimization of multi-step chemistry using a self-driven fluidic lab guided by reinforcement learning ↗
Amanda A. Volk, Robert W. Epps, Daniel T. Yonemoto, Benjamin S. Masters, Felix N. Castellano, Kristofer G. Reyes, Milad Abolhasani · 2023
Scientific rationale for path-dependent sequential decisions, beyond static one-shot optimization.
Scope: A real specialized platform and chemistry are involved; a lightweight synthetic environment cannot inherit its physical validation or performance.
- bennett2024
Autonomous reaction Pareto-front mapping with a self-driving catalysis laboratory ↗
J. A. Bennett, N. Orouji, M. Khan, S. Sadeghi, J. Rodgers, M. Abolhasani · 2024
Multi-objective final decisions and reward based on trade-off quality.
Scope: Specific catalytic experiments do not validate a generic multi-objective surrogate; performance depends on objective definitions.
- gravity2025
Gravity-Bench-v1: A Benchmark on Gravitational Physics Discovery for Agents ↗
Nolan Koblischke, Hyunseok Jang, Kristen Menou, Mohamad Ali-Dib · 2025
Physics-based mechanism shift and expert reference-policy precedent.
Scope: Upper-undergraduate gravitational tasks are distinct from biology or chemistry; success does not establish broad experimental competence.
- iqbal2024
Nesting Particle Filters for Experimental Design in Dynamical Systems ↗
Sahel Iqbal, Adrien Corenflos, Simo Särkkä, Hany Abdulsamad · 2024
Stateful trajectories and history-dependent experiments should be evaluated separately from independent reset queries.
Scope: Numerical results are restricted to tested dynamics; particle methods may be too expensive for the first demo baseline.
- discoveryworld2024
DiscoveryWorld: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents ↗
Peter Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Oyvind Tafjord, Peter Clark · 2024
Competitive context, task-versus-variation distinction, and separate measurement of procedural and explanatory performance.
Scope: Fictional low-fidelity worlds are explicitly proxies for scientific competence, not proof of physical discovery.
- gneiting2007
Strictly Proper Scoring Rules, Prediction, and Estimation ↗
Tilmann Gneiting, Adrian E. Raftery · 2007
Brier, log-score, CRPS or interval-score diagnostics when probabilistic submissions are supported.
Scope: Arbitrary clipping, nonlinear transforms or mixed objectives can alter propriety; no claim of strict propriety is made for an entire composite game reward.
- boxhill1967
Discrimination Among Mechanistic Models ↗
George E. P. Box, William J. Hill · 1967
Foundational candidate-disagreement acquisition for the kinetic demo.
Scope: Assumes candidate model structures and tractable uncertainty approximations; identifying one candidate does not establish that nature belongs to the candidate set.
- raue2009
Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood ↗
Andreas Raue, Clemens Kreutz, Tim Maiwald, Jasmin Bachmann, Marcel Schilling, Ursula Klingmüller, Jens Timmer · 2009
Reject impossible exact-parameter graders; prefer identifiable predictions or equivalence classes.
Scope: Profile likelihood itself is not a universal identifiability proof; every new task needs its own separation and sensitivity checks.
Synthetic research prototypes. The public demonstrations are inspectable examples, not protected evaluation episodes. Independent scientific review and real-world transfer have not been established.
Source inspection, deterministic grading, and a successful reference policy do not demonstrate that an environment improves a trained model. That requires a separate training and evaluation study.