Engineering & R&D.
Improve a frozen metric by changing one thing at a time, and keep only what holds up.
Engineering follows a different logic from fundamental research: there is no law to prove, but a measure to improve. Mutome changes one movable object (code, parameters, or a simulation setup), runs a frozen evaluation, and keeps or reverts the change under a rule fixed in advance.
Runs today: the keep-or-revert loop, on a bounded code-change campaign.
See an example
One change. One fixed test. A recorded decision.
Change one thing, run the same frozen test against a baseline with the same budget, and keep the change only if it wins under the rule fixed in advance.
Did this change actually help, or did we get lucky?
The problem
A change that looks better on one run may be noise, may break something else, or may simply be tuned to the test.
Why it is hard
If the test can be edited, or several things change at once, a better number tells you nothing about why. And a gain that needed more budget than the baseline is not a gain.
How Mutome approaches it
One movable object, a frozen evaluation command, a rule set in advance, and a baseline given the same budget: keep the change if it wins, revert it otherwise. Every attempt can be replayed.
One change, keep or revert
This is the loop Mutome ran on a bounded autotune campaign. The figure shows the mechanism, not a result.
What runs today
The git-backed loop records each candidate patch, the evaluation command, the metric, and the keep-or-revert decision, so every attempt can be replayed. It has run on a bounded code-change campaign; discarded attempts stay in the record.
Not yet
Parameter and simulation-setup campaigns, and any claim of a broader engineering gain, which would need budget-matched baselines on held-out workloads and an independent replay.