ADR-0013: Where input values come from, and how a campaign is put together

Status: Accepted, amended at M1.11, M2.7a, M2.4, M2.10a, M9.1, M9.3, M10.1 and M10.3, and on 10, 12, 13 and 25 October 2026 Date: 2026-09-13

Context

M1.5 shipped the first mechanism for choosing input values: a list of sources asked in order, where the first one to answer wins and the order is fixed inside the generator's constructor. It works, and it cannot express two things the next four increments need.

It fixes the source, not the value. If a parameter named name has an example in the document and examples are asked first, then a curated dictionary of fifty good names is dead weight for that parameter for the whole run. What is wanted is "the example most of the time, the dictionary the rest", and a list that stops at the first answer cannot say that.

The preference is global, when it belongs to the parameter. The order of the list is one decision that applies to every parameter of every operation. There is no way to say "for identifiers, prefer what we have actually seen; for everything else, prefer what the document states".

Four more sources arrive in M2 — the document's declared examples, values matching a declared format, a dictionary curated by hand, and an external program — and each of them would be written against whatever shape exists at the time. Settling the shape afterwards is four rewrites.

What the tools we are measured against actually do

Not what they advertise; what their documentation says about the mechanism.

Tool	How a value is chosen
RESTler	Two levels: custom payloads keyed by parameter name or by path inside the body, and per-type pools as the fallback. Every value in a pool is tried, not sampled
Schemathesis	Separate phases over the same schema — examples, coverage of boundaries, fuzzing, stateful. Not a priority order: a budget split
EvoMaster	Search. The value is a variable of an individual, mutated by an evolutionary algorithm with coverage as the fitness; examples and enumerations seed the initial population
CATS	--refData per path and field, a user dictionary, and a catalogue of per-field fuzzers
RESTest 1.x	One generator configured by hand per parameter in testConf.yaml. Without the file, there is no tool

Two observations decided this ADR. None of them uses a global priority chain as its main mechanism. And where there is priority at all — RESTler, CATS — the key is the parameter, not the position of a source in a list.

One measurement of our own

Of the eighteen findings the review of M1.5 produced, ten were in the arithmetic of inventing a value from a schema's constraints: an overflow on maxLength: 2147483647, an exclusive bound ignored when an inclusive one sat beside it, numbers that were never fractional. A dictionary lookup has no arithmetic to get wrong. That is evidence about where the cost of this component lives.

Decision

1. Sources stay behind one narrow interface

ValueProvider in restest-core does not change: asked about one value, either suggest something or say nothing. Eight sources, of which three exist today:

Source	Answers when
enumeration	the schema states a closed list
example	the document states an example for this value
format dictionary	the schema declares a format we have values for
default dictionary	always, for a type we have values for
custom dictionary	the user's file has an entry for this operation and parameter
observed values	this run has seen a value for a parameter of this name
fuzzing dictionary	always — values designed to be refused. Built at M2.7a
constraint-directed construction	always, as the last resort

The four dictionaries are one class with a different key: the JSON type, the declared format, the parameter's name, or the pair of operation and parameter. The file format is the same for all of them, and it is a published format (design principle 8).

2. A strategy is an ordered list of groups; within a group, sources are sampled

strategies:
  - name: nominal-custom
    share: 75          # amended at M2.7a: a proportion, not the `40m` first written here
    sources:
      - exclusive: [enum]           # if it answers, the choice is made
      - weighted:                   # ask all, sample among those that answered
          observed: 30
          example:  20
          custom:   40
          format:   10

An exclusive group stops at the first answer. A value outside a declared enumeration is invalid by construction and belongs to a strategy that intends to be refused, so the enumeration is never mixed with anything.

A weighted group asks every source, collects the answers, and samples among them. Every source is an in-memory lookup, so asking all of them costs nothing worth measuring.

The percentages must sum to 100 and are validated. They are an intention, not a guarantee: a source that has nothing to say does not get its share, which is redistributed among the sources that answered. Written down here because somebody will measure the outcome and find it does not match the file.

Weights live in the strategy, never in the source. A source that rates its own confidence is a magic constant with a friendly name; a percentage in a plan is a decision somebody made on purpose.

3. The test case records what was intended; oracles are not attached to it

A test case carries an intent: I believe these values are acceptable, I expect this to be refused, and here is what I broke, or I do not know. Oracles stay where ADR-0006 put them — outside, observing interactions — and read the intent to decide whether they apply.

Oracles are configured per run, and per operation from M3.4. They are not stored on the test case, although they could be. The reason is that these are facts and policy, and only the facts have to be recorded:

	How many values	How stable
Intent	about four	does not grow when an oracle is added
Oracle catalogue	about forty, and growing (M3.1 adds seven, M3.2 about twenty-five)	changes every milestone

An oracle reference stored in a run written today names something that may be renamed or removed tomorrow; 2XX_P is derivable from "I believed the values were acceptable", and the oracles for HTTP semantics apply whatever the intent was. Storing the smaller, stabler fact keeps restest recheck able to apply a different oracle set to an old run, which is the whole point of keeping runs.

The intent belongs to the test case rather than to the strategy because one strategy produces several intents: probing a documented limit expects the minimum to be accepted and the minimum minus one to be refused, in the same strategy, on the same parameter.

The third value, I do not know, is not a placeholder. It is the honest intent of a value invented from a schema, because an API may refuse it for a rule the document does not express. It becomes I believe this is acceptable when the value comes from a source that knows the API.

4. A deliberate violation is a mutation of a test case the API accepted

Instead of a generator that builds invalid requests from nothing, an operator takes a test case that actually returned a 2xx and changes one thing: drop a required parameter, send the wrong type, step outside a documented bound, break an enumeration, break a pattern, and later violate an inter-parameter dependency. The intent comes from the operator for free, and a 2xx afterwards is a finding attributable to that one change.

Probing documented limits is therefore an operator, not a strategy of its own.

Fuzzing every parameter at once stays as a separate strategy. It is a legitimate and common way to hunt for 5xx, and its intent records that a refusal cannot be attributed to any one parameter — which is exactly the difference between the two.

5. The pieces with a memory listen to the event stream; generation never reads the store

Two components need to know what has already happened: the dictionary of observed values, and the memory of accepted test cases that mutation works from. Both are listeners on the event stream (ADR-0006), keeping their own bounded index in memory. Neither queries the interaction store.

The store is for looking back. The event stream is for reacting.

This is not only about module boundaries — although it is also that, since restest-store may only be reached from restest-cli and this keeps the rule intact with no exception. It is what these two components actually want: an identifier observed forty minutes ago may have been deleted since, and a test case that succeeded early may not succeed now. They need recent, not complete.

It is also the answer to what ADR-0008 means by providers being "asynchronous by default". The interface stays synchronous and simple: a source answers instantly from an index that a listener fills in the background. An asynchronous interface would not even be better at the job, because it still lets a caller wait; a filled index cannot.

6. The scheduler is the only component that knows the budget

Amended on 25 October 2026: the only component that decides anything by it. The reports are told the budget too, to show how a run went over it (ADR-0038); see the amendment of that date.

Which strategy runs, how often, and for how long is decided in one place. Generation does not know what time it is, and there is no concept of a "phase" inside it: a phase is a share of the budget, and that is the scheduler's vocabulary.

The plan also carries the filters — the HTTP methods to exercise, whether to keep to methods HTTP calls safe, and named operations to restrict a campaign to. Filtering by method is not the same as "read-only": an API that searches with POST is ordinary, and one of the five specifications in our own corpus does exactly that. The words safe and idempotent are HTTP's own and mean something precise; "read" and "write" would not.

7. Randomness stays as a mechanism; the reproducibility promise is made per strategy

The seed is recorded with the run and can be set. What changes is what is promised:

Strategies	Reproduced by
without a memory — nominal from dictionaries, limits, fuzzing	the seed
with a memory — observed values, mutation	replaying the stored run

M1.5 promised more than that, in a Javadoc comment and in a pull request, and the promise breaks the day the dictionary of observed values arrives. Stating it per strategy is the honest version.

The seed is kept rather than dropped in favour of replay alone, for three reasons that replay does not cover. A generator whose randomness cannot be controlled has no deterministic tests. A failure that happens during generation leaves nothing in the store to replay, by design — ADR-0005 says a test case that never became a request produces no interaction — and that is not hypothetical: M1.5's overflow on maxLength: 2147483647 threw before any request existed. And M6.2's overhead regression test needs a fixed workload to compare one commit against another.

Random invention of values shrinks to one job: constructing a value that satisfies constraints no dictionary entry can satisfy, such as a string between twelve and fourteen characters or a multiple of seven between 1,000,000 and 1,000,010. When M2.3's walk over documented limits lands, that construction becomes deterministic and the random arithmetic goes away.

Consequences

• Adding a source of values is one class and one line in a plan. That is the property the next four increments were going to test, and it is why this is settled before they are written.
• Dictionaries replace arithmetic with lookups, in the component where the arithmetic produced ten of eighteen review findings.
• Provenance must name the source. With sampling among sources, the composition of a run is probabilistic, and without knowing which source answered, "did the dictionary I wrote by hand help?" cannot be answered at all — not in a report, and not by the feedback of M6, which would have nothing to learn from. Most of what is needed already exists: a value invented or looked up is recorded as generated by a named source, and a value read out of an earlier reply names the interaction it came from. The gap is narrower than it looks and sits in one place — a default and an enumerated value are both recorded as declared, with nothing to tell them apart, and the declared examples of M2.2 would be a third. ADR-0005's amendment considered separating them and left it, reasoning that a consumer can compare the value against the schema's own default and enumeration. That reasoning has a hole worth naming before M2.2 meets it: a schema whose default is also one of its enumerated values — default: available among [available, pending, sold], which is the common shape — leaves the two indistinguishable by comparison. Whoever adds the third case decides whether that is worth a sub-kind. Decided at M2.2: it is. A value the document stated now names which statement it was read from, and the name is optional so that a run recorded before the change still reads. See ADR-0019 §4.
• A varied dictionary has an arithmetic problem worth stating. With twenty entries and a parameter that wants an e-mail address, one draw in twenty fits; with three such parameters in one request, all three fit once in eight thousand. Two cheap mitigations, both in this decision: index on format when the document declares one — reading the specification, not inferring anything — and remember what produced a 2xx. Without one of them, a varied dictionary is a lottery for anything but a single parameter.
• Two increments already planned inherit obligations. M1.6 publishes the event stream and adds the intent to the test case — it published the event stream and did not add the intent, and the obligation moved to M3.1b, the first increment with an oracle that reads one — which is a store layout change — supported, since the file records the layout that wrote it. M1.7 keeps the loop in the command-line module, so that generation never gains a dependency on the engine or the store.
• Evaluation integrity. The default strategy must not depend on data curated for the five APIs the tool is measured on, and campaign results must report "as it comes" separately from "with a curated dictionary". Otherwise the comparison against tools that were given nothing is not a comparison. M8.2's ablation study is where this is reported, but the constraint is on the design, now.
• Licensing. A dictionary shipped with the tool redistributes whatever data is in it — names, postal codes, airport codes. That has to be checked before the first file is written, for the same reason the golden corpus is not published inside an artefact.
• More vocabulary than there was: source, dictionary, strategy, operator, intent, plan. Six words for what used to be one list. The defence is that five of them already existed in the roadmap under other names, and the sixth — intent — is what makes two of the oracles possible at all.

Left to M4, on purpose

Testing that spans several requests needs two decisions this ADR does not take, because taking them without a single sequence written would be guessing.

The unit of work. A sequence cannot be a precomputed list of test cases: the identifier in the second step comes from the first step's response, and in create, delete, read the expectation of the third step depends on the second having succeeded. So the scheduler's unit has to be something that can be asked for the next step given what has happened, rather than a test case. A shape was sketched during this discussion and deliberately not adopted: it had nowhere to put the clean-up of a resource created by a step that later failed, and it did not fit the way RESTler extends sequences across a whole run. Nothing in the current design prevents it — the loop lives in the command-line module and generation never sends anything — so M4 can choose the shape when it has a sequence to write.

Interference between concurrent sequences. One rule was proposed and refuted during this discussion, and it is recorded here so that it is not proposed again. The rule was "a sequence may only assert about resources it created itself". It fails because the dictionary of observed values hands the same identifier to everybody:

unit A   POST /pets           → 201 {"id": 7}        A created 7
                                                     the listener harvests 7
unit B   DELETE /pets/7                              B takes 7 from the dictionary
unit A   GET  /pets/7         → 404                  A reports a use-after-free that never happened

The replacement to start from, which makes ownership true instead of merely asserted: a sequence never borrows an identifier; it creates what it needs, and the dictionary of observed values serves only tests of a single request, where a stale identifier produces a 404 that no oracle treats as a finding. Its honest limit is that resources which cannot be created through the API — those needing another actor's permission — leave their sequences untestable, and reported as such.

Until M4, concurrency is safe and stays. Interference needs both a shared identifier and an oracle that asserts across requests, and neither exists yet. What would not be safe is making the loop sequential "just in case": against an API answering in 200 ms, twelve requests in flight send about sixty a second and one at a time sends five, and our own measurements would not notice, because idle time reports the tool's own waste and a sequential loop waiting on the API is never idle.

Alternatives considered

• Keep first-answer-wins as the only mechanism. What M1.5 has. It cannot mix an example with a dictionary for the same parameter, and it makes the order of the list a single global bet that cannot be right for every parameter at once.
• Confidence declared by each source — "I am 0.8 sure about this". Rejected: a number nobody calibrates against evidence. The tool that does use weights, EvoMaster, learns them from coverage rather than being told.
• Phases as a first-class concept inside generation, as Schemathesis has them. Rejected as a concept, adopted as an outcome: a phase is a share of the budget, and the scheduler already allocates budget. One noun fewer for the same behaviour.
• Oracles attached to the test case, as RESTest 1.x effectively did. Possible, and it is the same information viewed as policy rather than as fact. Rejected because the catalogue is large and moving while the fact is small and still, and a run stored today would name oracles that no longer exist.
• Semantic dictionaries — person names for name, airport codes for origin. Rejected as the starting point, because the hard part is not the list, it is deciding what a parameter means, and that is inference the tool cannot do reliably without the specification's help. Reading a declared format is not inference and is adopted instead.
• Generation reading the interaction store to find accepted test cases and observed values. Rejected: it wants recent rather than complete, and it would put the store inside the loop it is supposed to be a record of.
• Dropping the seed entirely, on the grounds that replay reproduces a run. Rejected for the three cases replay does not cover, listed above. The half that was right — that replay, not the seed, is the reproducibility mechanism a user is offered — is adopted.
• Deciding the unit of work for sequences now. Rejected as designing a mechanism before its first consumer exists, which this project has already done twice at a cost: an unused method added to ValueRequest, and a provider seam built and then left with no way to plug anything into it.

Open question

What a campaign does when nobody configures anything. The share of a budget between nominal, mutation and fuzzing is the one number here that cannot be argued into place: it is measured. When the pieces exist, one afternoon of campaigns against the five specifications in the corpus, with three different splits, answers it.

First number, M2.7a. Fuzzing takes a quarter and nominal the rest, in one constant, with --fuzzing <percentage> to change it. It is a starting point and is still unmeasured: the campaign this paragraph asks for needs a real API and has not been run. Mutation does not exist yet, so the split is between two things rather than three. M2.10 owns the answer, along with the file that makes the number something a person can set rather than something a build fixes.

Amendment (M1.11)

Date: 2026-09-16

The run's randomness comes from a generator java.base is required to carry, built directly rather than asked for by name. A seed therefore means the same run on every Java runtime, and RESTest starts on runtimes that ship only the compulsory part of Java.

Section 7 above says the seed is recorded and can be set. It did not say where the randomness itself comes from, and the code that implemented it chose by name.

Why

M1.5 asked for L64X128MixRandom, on reasoning written into the class: wherever "the same number produces the same test cases" holds at all, it has to hold on somebody else's machine as well as on ours, and asking for whatever the platform considers default would make it hold only between two runs on the same version of Java.

The reasoning was right. The name was wrong. On Java 21 that family of generators is not part of the compulsory Java; it is in jdk.random, a separate optional module. The official eclipse-temurin:21-jre image carries 49 modules and that is not one of them, and neither jlink nor a distroless base produces it unless asked. On such a runtime every run died in the generator's constructor, before a single request:

restest: the run could not be completed: java.lang.IllegalArgumentException:
No implementation of the random number generator algorithm "L64X128MixRandom" is available

and answered 4, RESTest itself malfunctioned. That code was right about what had happened and wrong about whose fault it was: somebody had pointed the tool at an ordinary JRE.

The release boundary is the part worth recording, because it explains how this survived nine increments and nine rows of continuous integration. Java 23 folded those generators into java.base; on 21 and 22 they are a separate module. Checked against the official images:

Runtime	L64X128MixRandom
eclipse-temurin:21-jre	absent — the failure above
eclipse-temurin:22-jre	absent
eclipse-temurin:23-jre	present, in java.base
eclipse-temurin:24-jre and 25-jre	present, in java.base

So the only runtimes where RESTest could not start were the two oldest it supports — 21 being the baseline every module compiles for and the version the whole project promises — while the machines it is written on run 25, where the mistake is invisible. Continuous integration tests 21, 25 and 26, but on full installations, which carry the generators on every release.

java.base is compulsory on every release, and on 21 it carries three generators - from 23 it carries the whole family as well, which is what makes this repairable at all. SecureRandom cannot be reproduced from a seed, which is the one property needed here. Random is the 48-bit linear congruential generator Java's own documentation calls legacy. SplittableRandom is the third, and it is the one now used.

What changed

new SplittableRandom(seed) in place of RandomGeneratorFactory.of(name).create(seed). Asking the factory for SplittableRandom and constructing it directly give the same sequence — checked, and the same values on Java 21 and 25 — so this removes a lookup rather than introducing a mechanism. That the lookup is gone matters twice over: it is what failed, and it is a service lookup on the startup path, which is the kind of thing that also needs arranging for wherever a runtime is assembled rather than installed.

What it costs

SplittableRandom is not the most modern generator Java offers. It is a good one for the two jobs it has here — choosing among a list, and filling in a number — and it can be split, which is what a future generator running on several threads would want. The trade is a generator that is merely good everywhere, instead of the best one on machines that happen to have it.

One thing this decision buys less of than it first appears. SplittableRandom's documentation promises identical sequences from identical seeds in the same program; the only generator in java.base whose algorithm the specification pins across implementations, "for the sake of absolute portability of Java code", is java.util.Random. So the cross-runtime half of the promise rests on what the implementations actually do rather than on what the specification requires. Measured, the same seed gives the same sequence on 21, 22, 23, 24 and 25; and a test now runs a seeded generation inside the container and the same one outside it and requires the requests to match, so if a release ever did change it, a build would say so rather than a user discovering it.

The algorithm is now a compatibility surface, like the exit codes in ADR-0015. A seed only means anything against the generator that consumed it, so changing the generator changes what every seed recorded anywhere means. That is a deliberate, breaking change to be made in another amendment here, not by editing a line — and "not by editing a line" is now enforced rather than asked for: a test holds the exact order a fixed seed produces from three operations that carry no parameters, so nothing but the source of randomness can move it. Swapping in java.util.Random fails it, which was checked.

The accidental version of the original mistake is ruled out separately, by an architecture rule: no production class may name a generator. It reports every of and getDefault in java.util.random, on the factory, on RandomGenerator itself and on each of its nested kinds — RandomGenerator.of(name) is the shortest way to write this bug and the first version of the rule did not see it. Holding a RandomGenerator is untouched, and so is RandomGeneratorFactory.all(): asking a runtime what it actually has is the careful thing to do and works everywhere, so reporting it would be reporting the remedy.

The rule is a static one, and static rules only see what is written down. A library RESTest depends on could reach for an optional part of Java without anybody here writing a line, so the rule is backed by a gate that runs the whole command inside the very image this was reported on, on every build that asks for containers. It establishes first that the image genuinely cannot provide the generator - by asking it for one, rather than by looking for the module's name, which is equally absent from Java 23 and later where everything works - so that it cannot pass by having nothing to prove.

A number printed by a run before this change names a different run now. Nothing keeps a seed beyond the line a finished run prints, so that is the whole of the cost; had the report file carried one, this would have been a change to a published format as well.

Nothing in section 7 changes. Which strategies a seed reproduces, and which need the stored run replayed instead, is unaffected by where the randomness comes from.

Alternatives considered

• Keep the preferred name, fall back to a java.base generator when it is missing, and record which one the run used. The obvious repair, and it re-creates in a milder form the very problem that naming an algorithm was meant to solve: --seed 12345 would mean one run on a full installation and a different run on a JRE. Recording the generator beside the seed makes that visible, not absent — whoever reproduces a container's run on their laptop still gets a different run, and now has to read a second field to find out why. A promise with a footnote about the reader's runtime is worth less than a smaller promise that holds everywhere, and the footnote would have been a moving one: the same command on the same image reads differently on 22 and 23.
• java.util.Random. The one generator whose algorithm the specification pins across every implementation, which is a stronger guarantee than the one chosen carries. Rejected on quality: it is a 48-bit linear congruential generator that Java's own documentation marks as legacy, and this is the component that decides what an API gets tested with. The guarantee it offers is bought back cheaply — by measuring across the five releases, and by the cross-runtime test named above — whereas its shortcomings could not be bought back at all.
• Raise the baseline to Java 23, where the problem does not exist. It removes this failure and nothing else, at the cost of everybody still on 21 — the long-term-support release most of the world runs. ADR-0003 chose 21 deliberately; a random number generator is not a reason to revisit it.
• Declare requires jdk.random in restest-gen's module descriptor. It changes nothing where the problem happens. RESTest runs on the class path, deliberately (ADR-0015), and module declarations are not read there. On a module path it would turn our own message into a resolution error from the JVM, which is worse to read and is not where anybody meets this.
• Fail with a message naming the runtime and saying what to install, and answer 3 rather than 4. A good answer to the question as it was asked, and unnecessary once the generator is one every runtime has: there is no such runtime left to report. Explaining how to install a missing piece of Java is a worse outcome than not needing it.
• Take whatever the platform prefers, which is what RandomGenerator.getDefault() does. Rejected in M1.5 and rejected again: it also names one of the optional generators, so it fails in the same place, and where it works it makes a seed mean whatever that runtime preferred.

Amendment (M2.7a)

Date: 2026-09-18

A strategy's share is a proportion of the budget, not a stretch of the clock. Every source is named, the last resort included. And the dictionaries these decisions describe now have a format.

Why

share: 40m in §2 is a duration, and it collides with the budget the user sets. A file asking for forty minutes has no sensible meaning under --budget 10m: cut it, ignore the budget, or refuse to run, and all three are worse than the question not arising. A proportion makes the same division serve a thirty-second check and the two-hour campaign M8 needs, which is what a file of strategies is for.

The sources listed in §1 are referred to by name in §2's plan, and one of them — "constraint-directed construction, always, as the last resort" — reads as though it were special. It is not. Being the last resort is where a strategy puts a source, not a property of the source, and treating it as ordinary is what lets a plan put something after it one day.

What changed

share is a percentage. The example in §2 now reads share: 75. Exclusive and weighted groups are unchanged as concepts, and so is the renormalising of weights among the sources that answered.

One of them is now built and one is not. The exclusive group is what the ordinary way of building a request is, and §2's exclusive: [enum] is honoured exactly as written: the closed list of values a document says it accepts is asked first and nothing may override it, a dictionary somebody wrote included. The weighted group is not built, and ADR-0020 §5 records why, along with the competitor this increment created and the advice that stands until 2.4 brings it.

One thing §2 says needs reading carefully now that both numbers are percentages, because it is true of one of them and not the other. Weights within a group must sum to 100 and are validated, as §2 says. Shares between strategies are counted against each other and need not. They divide one budget between all the strategies there are, so three strategies asking for 1, 1 and 2 divide it into quarters exactly as 25, 25 and 50 would; requiring a total would be requiring arithmetic of somebody for no gain. The implementation relies on that: it multiplies one side rather than dividing the other, so that asking for a quarter is a quarter whether one list of awkward values is in play or thirty.

The file format the five dictionaries share is ADR-0020. It is YAML, it adds a fifth keying this record missed — the name of the shape, which is how a whole request body is indexed — and it deliberately carries no statement about what the API will make of its values, because a file of values cannot know: which lists a strategy draws on is named in the plan, as §2 already has it.

What it costs

Nothing yet, because no file of strategies exists to be rewritten. Recorded now because it is cheaper to amend a sentence than a format.

Amendment (M2.4)

Date: 2026-09-20

The format dictionary of §1 is not built. What a document says about the characters of a value is read where what it says about the length already is.

What changed

§1 lists eight sources, and one of them - "format dictionary: the schema declares a format we have values for" - was to arrive at M2.4. It has not, and will not as a source. A value of the kind a document names is now built by the same code that reads minLength and maxLength, alongside a value spelled the way a stated pattern demands. The chain is untouched: no source was added, none moved, and nothing about the order in §2 changed.

The reasoning is ADR-0022 §2, and the short version is that a dictionary sees only its key while the shape states its kind, its spelling and its lengths at once, and all three have to hold together.

Seven sources, then, of which four exist. The format keying of ADR-0020 is untouched and is still one of the four dictionaries this record describes - it is simply a file a user writes rather than one RESTest ships.

What it costs

The measurement in §2's note about a varied dictionary - "index on format when the document declares one" - is still the right answer for somebody's own list, and is still what the keying is for. What this record loses is the shipped file that would have exercised it, and ADR-0020 §5 loses the competitor it was counting on to make weighted groups worth having. M2.10 is unchanged in scope.

Amendment (M2.10a)

Date: 2026-09-22

The file §2 sketched is built, and three of its ideas turned out to be saying what the structure already said. §6's safeOnly is answered by methods instead. ADR-0023 is the record; this notes where that file departs from what was written here, so that a reader of §2 and §6 is not surprised by what they find on disk.

No exclusive group. §2 writes one beside weighted, but an exclusive group means ask these in turn and stop at the first answer, which is what an ordered list of sources already means. The keyword said nothing the order did not, and the plan RESTest ships needed four single-entry groups in a row to say nothing at all. weighted: is the only group keyword. The semantics §2 decided are untouched.

No custom or format among the source words. §2's example names five sources, of which two are dictionary keyings rather than sources. A list of values is named by naming the list - dictionary: good-values - and a plan says which kind of thing each name is, so that a file called example cannot silently shadow the tool's own samples. The words that remain are enum, example, default and random, with observed joining them at M2.5b, which is the source this whole increment was taken early to make nameable.

No safeOnly. §6 asks for it by name. HTTP's safe methods are exactly GET, HEAD, OPTIONS and TRACE, so methods: already says it, and keeping both would force a rule for a file that sets both. The point §6 was really making survives and is why the key is not called readOnly: an API that searches with POST is ordinary, and one of the five in the priority corpus does exactly that.

A strategy serving only some kinds of operation is not built. §6 puts the filters in the plan, and the run-level one is there. A filter per strategy - so that writes could be pushed at harder than reads - was designed, written and dropped before it shipped: its one real use is that split, and it was the sole cause of shares that summed to 175, of redistribution, of per-operation validation and of operations no strategy served. A strategy without a scope already means every operation, so adding it later breaks no file anybody has written.

§7's promise is untouched and still ahead of us. Every strategy this increment can build is reproduced from the seed, because none of them has a memory yet. M2.5b is where the table in §7 starts to matter.

Amendment (M9.1)

Date: 2026-09-23

§6's scheduler is built, and generation still does not know what time it is. Scheduler holds the deadline and the order requests go out in, opens a run with a first round that sends every operation once, and asks generation for each request. What §6 calls a phase - a stretch of the budget, in the scheduler's vocabulary - is now announced on the event stream, and the first round is the first of them.

The share between strategies is still drawn per request, inside generation, as M2.7a left it. Honouring a share as a stretch of the clock was row 2.10b, absorbed into 9.1 and then set aside for 2.1: it moves none of the competition's measurements, and a strategy chosen by the clock would break §7's promise for every plan with no memory. ADR-0026 §7.

Amendment (M9.3)

Date: 2026-09-26

The two decisions "Left to M4, on purpose" were built, measured, and not merged, so they stay open. Roadmap row 9.3 needed both, because M4 is now after v2.0. The code is kept on the branch experiment/m9-3-sequence-pairs; this records what it did, what it measured, and what whoever takes the decisions next - 10.3, or 4.4 after v2.0 - starts from.

What was built

The unit of work was a pair: a creation and the request it was made for. The scheduler sent the creation in the other request's place in the round; the loop handed the creation's answer back the moment it arrived; and the other request went next, taking its identifier from its own creation's reply, never from the memory - this record's rule, a sequence never borrows an identifier; it creates what it needs, taken literally. Gaps the two addresses share in the same places were filled with what the creation was sent there, so a pet asked for belonged to the owner it was made under, and each value named the exchange it came from. A pair was made only in the ordinary rounds, only for a plan with a memory, and only when the memory had nothing for a gap that a POST at the same address - or, for a gap named like an identifier, about the same kind of thing - creates. One creation of each kind was awaited at a time, and one that made nothing usable was not sent as one again until the next round.

Interference was accepted rather than prevented, with the engine's concurrency left on: a 404 on the second request is not by itself evidence of anything, which costs nothing the competition scores and is why 4.5's stateful oracles still wait.

The memory forgot what a 2XX DELETE removed, so that "every identifier carried has since been deleted" was something a run could know.

What was measured

Four priority APIs from the benchmark's own images, restarted before every run, five seeds, sixty seconds, in three arrangements: both on, pairs off, and both off. 7 pairs fired in 338,229 requests, and neither lever moved operations covered, the area under that curve, branch coverage or distinct server failures beyond one standard deviation. The table, and the three findings behind it, are in the roadmap's 9.3 notes. The one that matters here: every kind of thing these APIs create also has a list, the list refills the memory every round, so "the memory has nothing" is almost never true - and a static reading of the corpus and the 2026 edition found one API in 52 where it would be.

What this changes about the two decisions

Nothing is decided. What the measurement adds is that the trigger, not the unit, is the open part: a sequence that waits for the memory to be empty waits for ever on APIs with lists. The shape built here - a pair, its second step reading its own first step's reply - worked where it fired, and is the natural unit for 10.3's operators, which create their own victim and so do not wait for anything to be missing. Forgetting is not part of that unit and needs its own case: a 2XX to a DELETE is not always a deletion, and where the memory is thin, forgetting leaves invention in the place of a stale value.

Amendment (M10.1)

Date: 2026-09-27

§3 and §4 are built, and the intent has four values rather than three. ADR-0027 is the record; this notes where what was built departs from what was written here.

§4: a strategy says mutates: accepted. Its requests are a request the API accepted with one thing changed, drawn from a memory of accepted requests that listens to the event stream, as §5 said. Eleven operators, where §4 named five and the roadmap nine: "oversize" and "send the empty value" each became two, one that breaks what the document states and one where the document says nothing, so that the intent each records is true.

§3: four intents. Acceptable, refusal expected, unknown, and a fourth, pushing, for the requests of the strategy §4's last paragraph describes, whose refusal cannot be put down to any one value. Acceptable exists and is not yet given to anything; ADR-0027 §2 says why. The consequence below that calls the intent "a store layout change" was wrong: the test case is stored as JSON, and the intent and the change it records are two more members of it.

The unit of work for sequences is still open. A change to an accepted request is one request, not a sequence, and the two decisions "Left to M4, on purpose" are untouched by it.

Amendment (M10.3)

Date: 2026-09-28

The two decisions "Left to M4, on purpose" are taken, by ADR-0028, which records how.

• The unit of work is a series: a fixed list of steps about a thing the series created, each built once the answer to the one before is in, one step of it in flight at a time.
• Interference between concurrent series is accepted rather than prevented, as the M9.3 amendment had it.
• The trigger is no longer the open part: a series starts when the plan's strategy that sends them is drawn for a creation, and waits for nothing to be missing.

The rule this record proposed holds: a sequence never borrows an identifier; it creates what it needs. A series asks about the thing its own creation made, as far as the reply lets anyone tell. Its identifier is read from the creation's reply or its Location header, never from a list the reply holds nor from what the creation was sent in its own address, and a POST whose address ends in a gap is only sent twice: it names a thing that already exists, so nothing is asked about it. Two things the reply cannot settle. A thing the creation names itself: when the name it was sent is one the memory supplied - petstore's POST /pet with a remembered id, its POST /user with a remembered username - the series is about whatever that POST wrote. And a POST that attaches a thing which already exists and answers with it, which reads like one that makes a thing under another; GitHub answers its attachments with lists, which are never read for an identifier. The gaps before the thing's own take what the creation was sent, and neither memory hears a step, so nothing a series learns reaches another request. The honest limit this record named holds too: a thing whose creation needs something the run cannot make - a kafka cluster, which only exists - is created under one the memory supplies.

§7's table gains a row. A plan that sends series is reproduced by replaying the stored run, like one with a memory, because a series builds its later steps from its earlier answers. Its choices - which series, which step comes next - draw on numbers of their own, so a series never moves the ordinary rounds' numbers on. Where a creation has nothing optional in it, the ordinary rounds are the ones the same seed draws without series; where it has, the turn that begins a series leaves the optional parts out rather than drawing which to send, and the ordinary requests after it draw different numbers.

Amendment (10 October 2026)

Two rows of the table above, checked against each tool's code.

• RESTler (commit 6d984dee): every value in a pool is tried only in its bfs, bfs-fast and test-all-combinations modes, and only up to max_combinations, 20 by default. Its fuzz command runs bfs-cheap by default, which, like random-walk, bfs-minimal and its test mode, stops at the first combination of values that is accepted.
• EvoMaster (release 6.2.0, and its development branch at commit b72feb25): values are mutated by an evolutionary algorithm with coverage as the fitness in its white-box mode. Its black-box mode, the default since 6.0.0 and the mode it is measured in, uses SMARTS, a form of random sampling in which nothing is mutated. The same holds for "the tool that does use weights, EvoMaster, learns them from coverage", among the alternatives considered: true of its white-box mode only.

The point the table makes, that where tools have a priority it is keyed by parameter name rather than by source, still holds.

Amendment (12 October 2026): one keying for a file

Date: 2026-10-12

Since ADR-0033, a dictionary file is keyed by operation and place and by nothing else. The format keying the M2.4 amendment calls "a file a user writes" can no longer be written, and neither can type, schema or name. The list of awkward values RESTest carries keeps its keying by kind of value, as its own.

Amendment (13 October 2026): version 2 of the plan

Date: 2026-10-13

ADR-0034 writes the plan in a version 2 of its format. The M10.1 amendment's mutates: accepted is now kind: mutation, and a strategy pushes by being of kind fuzzing rather than by naming dictionary: fuzzing. The plan a run follows is unchanged.

Amendment (25 October 2026): the reports are told the budget

Date: 2026-10-25

The announcement that a run has begun carries its budget, so that report.json can cut the run's time into equal slices and say what came back in each (ADR-0038). §6's point stands: the scheduler is still the only part of the tool that decides anything by the budget - which strategy runs, when the run stops - and generation still does not know what time it is. The reports only show it.