Automated Testing
of REST APIs
For humans, for agents
Coding agent
Use RESTest to test the REST APIs you develop in this project. Follow the setup and usage instructions at https://isa-group.github.io/restest-site/docs/for-coding-agents/Paste it into your favourite coding agent: the agent installs RESTest and runs it for you.
Homebrew
brew install isa-group/tap/restestOn macOS and Linux. It installs the Java RESTest runs on, too.
jbang
jbang app install restest@https://isa-group.github.io/jbang-catalog/jbang-catalog.jsonOn macOS, Linux and Windows. It downloads a Java when there is none.
Docker
docker run --rm -v "$PWD:/work" ghcr.io/isa-group/restest:2.5.0 versionNothing to install but Docker. The manual says how to use it as a command.
From an API schema to the faults, with nothing to configure.
Give RESTest an OpenAPI document and the address of the API it describes. It makes up requests, checks every reply against what the document promised, and reports each fault with a curl command that sends the request again.
- It reads the document. OpenAPI 2.0, 3.0 or 3.1. It never looks at the API's code, so the API can be written in any language and run anywhere.
- It makes up requests and sends them for as long as you allow: a minute by default.
- It checks every reply against what the document promised, and says what to fix for each kind of fault.
- errorInternal server error (F100)POST /api/owners → 500The API answered 500, so it fell over while handling this request
- errorInvalid data accepted (F206)POST /api/users → 201The API accepted a request it should have refused: sent 0.5 for body.username, declared as a word
- warningUndeclared status code (F920)GET / → 302The API answered 302, which the document does not declare for this operation; it declares 200
- warningUndeclared content type (F921)PUT /api/visits/{visitId} → 404The API answered 404 as application/problem+json, which the document does not declare for that status; it declares application/json
- warningReply off schema (F200)GET /api/pets → 200The body does not match the shape the specification declares for it, answering 200 as application/json
Detailed results at a glance.
A run writes report.html beside report.json: one file that opens in any browser, with no network, in light and dark.
Summary and timeline
A fault, in full
What was found
A real run: Spring PetClinic REST, as the Web Fuzzing Dataset builds it, tested for a minute by RESTest 2.4.0.
52 APIs, a minute each.
The APIs of the Web Fuzzing Dataset, each started on its own and tested by RESTest 2.5.0. Every row of the showcase opens that run's report.
See the showcase| Most errors | Requests | Errors | Warnings |
|---|---|---|---|
| pay-publicapi | 78,240 | 77,522 | 77,814 |
| swagger-petstore | 152,376 | 61,733 | 8,919 |
| erc20-rest-service | 61,759 | 55,671 | 5,680 |
| person-controller | 37,170 | 20,242 | 27,622 |
| features-service | 30,178 | 17,399 | 2,066 |
Your coding agent tests as it builds.
Ask your coding agent to use RESTest while it develops your API: RESTest finds the bugs as they appear, and the agent fixes them before they reach you.
RESTest and coding agentsUse RESTest to test the REST APIs you develop in this project. Follow the setup and usage instructions at https://isa-group.github.io/restest-site/docs/for-coding-agents/- It installs or updates RESTest, with Homebrew, jbang, the image or the archive, whichever fits.
- It makes a quick first pass, a run of one minute, and tells you in a few lines what it found.
- It fixes what was found, when developing the API is its task, and checks each fix by sending the request again.
- It offers a deeper pass, with a dictionary of values it writes from the API's document, code and data.
Fit a run to your API. Three inputs, each optional.
A run needs nothing but the document. Each of these makes it fit your API better, and has a chapter of the manual.
--campaignA planHow a run builds its requests, where their values come from, and which operations it may touch.
--dictionaryDictionariesValues you know are good: identifiers that exist, names the API holds. A coding agent can write them from the API's document and code.
--setSettingsHow the tool itself behaves: how many requests at once, how long to wait for a reply, and switches that each turn off one thing a run does.
Do you need help? Contact us.
RESTest is developed by the SCORE Research Lab at the University of Seville (Spain). If you would like to try it on your organisation's APIs and need help, let us know, we will be glad to help.