The JSON Schemas of a dictionary and a plan

RESTest reads two files that a person, or a program, writes for it: a dictionary, handed over with --dictionary, and a plan, handed over with --campaign. Each has a JSON Schema: a file, itself written in JSON, that describes the shape the other file has to have — which members it holds, what kind of value each one is, which words are allowed where. A validator is a program that reads a file and a schema and says whether the file has that shape, and where it does not. Both schemas are written in draft-07, the version of the JSON Schema standard that editors, validators and the tools that guide a language model all read.

A schema is help, never a requirement. RESTest reads a file that names no schema, and checks every file it reads itself.

The third file a run can be handed, a file of settings, has no schema. RESTest checks one exactly without testing anything — restest run --settings <file> --print-settings answers 0 and prints every setting with where its value came from, or 2 with what is wrong in the file — and a schema would check less: a setting may be written as a number or as text, a length of time as 2m or PT2M, and one setting is checked against another, which a schema cannot say.

Where they are

Where	The dictionary's	The plan's
On the web, for the version you run	https://raw.githubusercontent.com/isa-group/RESTest/v<version>/schemas/dictionary.schema.json	https://raw.githubusercontent.com/isa-group/RESTest/v<version>/schemas/campaign.schema.json
In a checkout of the repository	schemas/dictionary.schema.json	schemas/campaign.schema.json
In a release's archive	restest-<version>/schemas/dictionary.schema.json	restest-<version>/schemas/campaign.schema.json
In the container image	/opt/restest/schemas/dictionary.schema.json	/opt/restest/schemas/campaign.schema.json
Installed with Homebrew	$(brew --prefix restest)/libexec/schemas/dictionary.schema.json	$(brew --prefix restest)/libexec/schemas/campaign.schema.json
Run with jbang	None on disk: the address on the web	None on disk: the address on the web

From 2.2.0 on, every release carries the schemas of the formats it reads, and each schema's address on the web is its file at that release's tag. Use the ones of the version you run: a later version may read a format an earlier one does not.

To copy one out of the image:

docker run --rm --entrypoint cat ghcr.io/isa-group/restest:<version> /opt/restest/schemas/dictionary.schema.json > dictionary.schema.json

In an editor

An editor that understands YAML through the YAML language server — Visual Studio Code with its YAML extension, among others — checks a file against the schema a comment on its first line names, and offers the members and the words allowed as you type:

# yaml-language-server: $schema=https://raw.githubusercontent.com/isa-group/RESTest/v<version>/schemas/dictionary.schema.json
version: 1
name: clinic
keyedBy: operationAndParameter
values:
  listOwners:
    lastName: [Franklin, Davis]

RESTest reads that line as the comment it is. A file written in JSON has no comments, and a $schema member inside it is refused like any other member RESTest does not know, so a JSON file is matched to its schema in the editor's own settings instead.

With a validator

Any validator that reads draft-07 will do. With check-jsonschema, from a checkout, the dictionary of the manual is checked with

check-jsonschema --schemafile schemas/dictionary.schema.json clinic.yaml

which answers

ok -- validation done

With the list under listOwners changed to lastName: Franklin, one value where a list of values belongs, it answers

Schema validation errors were encountered.
  clinic.yaml::$.values.listOwners.lastName: 'Franklin' is not of type 'array'

A plan has one place where a validator says more than one thing. A line of sources: is written one of three ways — source:, dictionary: or weighted: — and a validator, given a line that is none of them, tries all three and says why each does not fit. The plan restest run --print-campaign > plan.yaml writes is valid; with its fuzzing strategy's - source: random changed to - source: exmaple,

check-jsonschema --schemafile schemas/campaign.schema.json plan.yaml

answers

Schema validation errors were encountered.
  plan.yaml::$.strategies[3].sources[0]: {'source': 'exmaple'} is not valid under any of the given schemas
  Underlying errors caused this.

  Best Match:
    $.strategies[3].sources[0]: Additional properties are not allowed ('source' was unexpected)
  Best Deep Match:
    $.strategies[3].sources[0].source: 'exmaple' is not one of ['example', 'random', 'observed', 'dictionaries', 'default', 'enum']

  3 other errors were produced. Use '--verbose' to see all errors.

The line that says what is wrong is the one that goes deepest into the file, here the last: the word is not one of RESTest's. The others say that the line is not a dictionary: or a weighted: either. The output above is check-jsonschema 0.36.2's.

For a language model

The engines that hold a language model to a schema while it writes — so that whatever it writes has the shape — are what these schemas were checked with on 13 October 2026, by hand:

• llguidance 1.3.0 and outlines-core 0.2.14 were given each schema, accepted every whole example on these pages, and refused files with a member, a word or a kind of value wrong.

• llama.cpp: the converter that turned a schema into its grammars in Python, at commit c5a5535e of 3 September 2026, its last before llama.cpp kept only the one built into its server. Its grammars, matched by llguidance, did the same. The converter in the server was not run.

• XGrammar was not run. Its converter's code, at commit f294c51e, was read, and handles every part of JSON Schema these two use.

• Outlines does not hold a number to the smallest and largest a schema allows, so a share of 0 or a weight of 101 gets through it. RESTest refuses either when it reads the file.

• A model held to a schema writes the members in the order the schema lists them, and llama.cpp has it write the required ones first, so a dictionary's description comes last. The order means nothing to RESTest.

• The strict modes of hosted model services refuse both schemas, because both hold members whose names cannot be listed in advance: the operations of a dictionary, and the dictionaries of a group. A model there can still be given the schema to read with its instructions.

What a schema cannot say

A file valid by its schema is not always a file RESTest reads. A schema describes the shape of one file, and some of what RESTest checks is not shape.

RESTest refuses, although the schema lets it through:

• shares of a plan that do not add up to 100, or weights in one group that do not;
• a group with fewer than two sources in it — weighted: {}, or dictionary: {} inside a group, among them;
• two strategies with one name, and a name made of nothing but spaces, for a strategy or for a dictionary a plan names, or of nothing at all for a dictionary inside a group (dictionary: {"": 40});
• fuzzing as a dictionary's name, or as the name of a dictionary a plan draws on: it is the name of RESTest's own list of values to push with.

No schema sees a key written twice in one place, which YAML allows: the YAML is read before the schema is asked, and RESTest refuses such a file as it reads it. check-jsonschema refuses it the same way.

RESTest reads, and says as the run starts:

• an operation, in a dictionary or under a plan's only:, that the API does not have;
• a dictionary a plan draws on that nobody handed over.

The schema is stricter in one place: an HTTP method under a plan's methods: is written in capitals or in small letters, and RESTest reads Get as well.

When a validator reads YAML differently

RESTest reads in a YAML file only what JSON itself has — true, false, null and numbers — and everything else as text (the dictionary format says why). A validator that reads YAML by YAML 1.2's own core rules reads more, among it ~, Null and NULL as null, True, TRUE, False and FALSE as true or false, 0x1 and 0o7 as numbers, and .inf and .nan as numbers no JSON has. So the two can disagree about one file: version: 0x1 is valid to such a validator and refused by RESTest, which finds text where a version belongs, and name: True is the other way round. Quote such a value — "~", "True" — and both read it as text.