Testing REST APIs

Every chapter so far has assumed a response is correct without saying how anyone would know. verify.sh has been asserting status codes and schema shapes since Chapter 4, which already answers part of the question, but it has never named the layers underneath that one script, or said which of them a reader should write for their own API, in what order, and at what cost. This chapter names them. The example it documents (examples/24-testing/) trims Chapter 4’s booking resource down to just POST/GET /bookings — the subject here is the testing layers wrapped around that surface, not another pass at the resource model — and builds a real instance of each layer in both stacks: a contract check against the shared schema, a @QuarkusTest class and a pytest suite, a Postman collection run with Newman against both live services, and a tiny bounded load run on top of the same collection. None of these layers is hypothetical. All four exist as runnable code in the example, and the chapter’s job is to explain what each one buys, what it costs, and which of them a CI pipeline should actually run on every push.

The test pyramid for a REST API

The classic test pyramid — many fast unit tests at the bottom, fewer integration tests above them, a thin sliver of end-to-end tests at the top — translates directly onto a REST API, with one addition this book’s whole premise makes almost mandatory: a contract layer sitting under everything else, because a REST API’s correctness is measured against a document, not just against its own test suite’s expectations.

At the bottom, contract tests validate a live response’s shape against the schema the API promised to serve — not against hand-written assertions a test author guessed at, but against contracts/schemas/shipping.schema.json’s $defs, the same file every chapter since Chapter 4 has treated as the one place a field’s shape is decided. These are the fastest tests in the pyramid and the cheapest to run: no JVM boot, no ASGI app construction, just an HTTP call and a JSON Schema validation against a document that already exists. They’re also the least expressive — a contract test can tell a reader “the response has the right shape” and nothing about whether the service computed the right value inside that shape.

Above that sit integration tests — @QuarkusTest and pytest, run in-process against the application itself rather than over a live container’s network stack. These cost more than a contract check (a JVM has to boot, CDI has to wire, a pydantic model has to validate) but buy back real behavioral assertions a schema check can’t: that a 400 carries a Problem body with the right status field, that a quote computes to the exact decimal Chapter 4’s formula predicts, that a 404 for a nonexistent booking id returns the right title. They run fast enough, and often enough, to belong in a developer’s inner loop (on save, on mvn test, on uv run pytest) well before a push ever reaches CI.

End-to-end tests sit above that: a real client — in this chapter’s case, Newman driving the vendored Postman collection — exercising both live, fully deployed stacks over the network, exactly the way an actual caller would. These are slower and more expensive than either layer below them (a full docker compose up, two running services, a Node-based test runner), but they’re the only layer that proves the whole deployed system, not just a schema shape or an in-process unit, answers correctly on the wire.

At the very top, a thin load layer proves the system holds up under more than one request — not a statement about peak throughput or production capacity, but a cheap, deterministic sanity check that running the same request thirty times in a row doesn’t degrade correctness. The trade-off compounds going up the pyramid: speed and cost get worse at every layer, while the confidence each layer buys shifts from “the shape is right” to “the behavior is right” to “the deployed system is right” to “the deployed system holds up under repetition.” No single layer replaces another — a contract test catches a shape regression an integration test’s hand-written assertions never thought to check, and an integration test catches a logic bug a contract test’s structural validation is blind to by design.

Contract tests: the living schema as the test oracle

A contract test’s defining property is that it has no independent opinion about what “correct” means: it defers entirely to a document someone already agreed to. examples/24-testing/verify.sh makes that oracle explicit rather than implicit: Step 3 drives a POST/GET/404 sequence against each stack and pipes every response body through examples/_lib/validate_response.py, the same schema-validation helper every chapter since Chapter 4 has used, checking the full strictness a shared $defs entry carries — additionalProperties: false, every required field present, every pattern matched, nothing extra tolerated. A booking missing createdAt, or carrying an extra field nobody declared, fails here regardless of what status code it returned, because the contract test’s question isn’t “did the call succeed” but “does this body match what the schema says a Booking or a Problem is allowed to look like.” Before trusting that gate on a real response, Step 1 of verify.sh runs validate_response.py --selftest — a check against known-good and known-bad fixtures, proving the validator itself has teeth before a single live response is handed to it. A schema gate nobody has verified can reject a bad shape is a gate that might be silently passing everything.

The same oracle-not-opinion idea extends past this chapter’s own REST surface to the event contracts Chapter 21 introduced. A REST response validates against #/$defs/Booking in the shared schema; an event published to shipping.bookings.lifecycle validates against #/$defs/BookingEvent from that same file, and the CloudEvents envelope wrapping it validates against the fixed set of context attributes CloudEvents requires — specversion, id, source, type, time, datacontenttype. Both are contract tests in exactly the sense this chapter means: a consumer or a CI job checks a live artifact against a document agreed on in advance, not against assumptions baked into the test itself. A REST contract test and an event contract test differ in what they’re pointed at (a synchronous HTTP response versus an asynchronous message on a topic) but not in what question they’re answering.

This chapter’s own collection layers a second, looser version of the same idea on top of the strict $defs check: the Newman collection’s inline pm.response.to.have.jsonSchema(...) assertions, carrying small, hand-written JSON Schema fragments scoped to each request — a Booking schema asserting required on id/status/origin/destination/cargo/createdAt plus a handful of pattern and enum constraints, and a smaller Problem schema asserting just title and status. These stay looser than validate_response.py’s full $defs check (no additionalProperties: false, no full coverage of every field the shared schema defines) because they exist to make the collection self-describing to anyone reading the Postman file cold, not to replace the strict gate already running earlier in verify.sh. A reader opening the collection sees, inline, exactly what shape each request expects back, without needing to cross-reference the shared schema file at all. Two schema checks on the same response isn’t redundant work; it’s the strict oracle proving the contract holds, and the readable oracle proving the test suite itself documents what it’s checking.

Integration tests in both stacks

Contract tests prove a response’s shape; they say nothing about whether the logic that produced it is right. examples/24-testing/quarkus/src/test/java/.../BookingResourceTest.java and examples/24-testing/python/tests/test_bookings.py close that gap, each a real, runnable test file rather than a stub standing in for one. The Quarkus side uses @QuarkusTest, which boots the full application (CDI, JAX-RS routing, Jackson serialization) inside the same JVM as the test process, then drives it with RestAssured’s fluent given().when().then() style, exactly as if an HTTP client had called it, but with no socket crossing a network boundary. The Python side mirrors that shape with FastAPI’s TestClient, which runs requests straight against the ASGI app object in-process (no uvicorn process, no port, no container) using the exact same httpx-backed call surface a real client would use.

Both suites assert on real behavior a contract test can’t reach: the create test checks quote.amount computes to the exact string "3001.25" for a known cargo weight, not just that quote.amount is a string matching the right pattern; the 404 test checks the Problem body’s title reads "Booking not found", not just that some Problem-shaped body came back; the 400 test checks a request missing cargo gets rejected before a booking is ever created. These are the layer where “the service does the right thing” gets tested, as distinct from “the service’s response looks like the right shape.”

@Test
void createBooking_returns201WithLocationAndBookingBody() {
    given()
            .contentType("application/json")
            .body(VALID_BOOKING)
            .when().post("/bookings")
            .then()
            .statusCode(201)
            .header("Location", matchesPattern("^/bookings/bkg_[0-9A-HJKMNP-TV-Z]{26}$"))
            .body("id", matchesPattern("^bkg_[0-9A-HJKMNP-TV-Z]{26}$"))
            .body("status", equalTo("PENDING"))
            .body("quote.currency", equalTo("USD"))
            .body("quote.amount", equalTo("3001.25"));
}

def test_create_booking_returns_201_with_location_and_booking_body(client):
    resp = client.post("/bookings", json=VALID_BOOKING)

    assert resp.status_code == 201
    assert BOOKING_ID_RE.match(resp.json()["id"])
    assert resp.headers["location"] == f"/bookings/{resp.json()['id']}"
    body = resp.json()
    assert body["status"] == "PENDING"
    assert body["quote"] == {"currency": "USD", "amount": "3001.25"}

Neither suite runs inside verify.sh. That’s a deliberate boundary, not an oversight: booting a JVM and the full Quarkus test harness is a real cost, and paying it on every CI matrix leg, for every push, on top of the live-container contract and Newman layers verify.sh already runs, would make the bounded, fast CI path this book has built since Chapter 4 noticeably heavier for a layer that’s already proven correct locally. mvn test and uv run pytest are both fast on a developer’s machine — the JVM boot that would be expensive multiplied across a CI matrix is a non-issue run once, locally, before a push — so these layers live at build time and in a developer’s own workflow rather than in the gate every reviewer waits on. The boundedness discipline this book applies to Kafka waits and HTTP timeouts applies here too, just aimed at a different resource: CI’s job is the live, deployed, cross-stack assertions nothing else can make; a JVM or interpreter boot per test run belongs wherever that cost is already being paid anyway.

End-to-end with Newman

Contract tests and integration tests both stop short of proving the actual, deployed system answers correctly over the network a real caller would use. collections/shipping.postman_collection.json closes that gap: a Postman collection, vendored and adapted from cloud-native-design-patterns/examples/28-newman’s harness, carrying a Functional folder with four requests — create a booking and check 201/Location/schema, fetch it back and check 200/schema, submit a booking missing cargo and check 400 with a Problem body, fetch a nonexistent id and check 404 with a Problem body. verify.sh runs that folder with npx --yes newman@6, once against the live Quarkus container on port 8103 and once against the live FastAPI container on port 8023, requiring zero failed assertions on each run before the gate passes — the same collection, unmodified, pointed at two independently implemented stacks sharing one contract, which is itself a quiet proof that the contract constrains both implementations rather than one of them merely happening to pass.

What makes this layer end-to-end rather than a second contract check is what it proves beyond schema conformance: an actual HTTP round trip through the real framework stack, real routing, real serialization, real error handling, exercised over the network the way a production caller would hit it, with assertions covering both the happy path and the negative cases a contract test alone might never think to drive. verify.sh doesn’t just read Newman’s exit code — it parses the summary table’s assertions row and fails loudly if the failed count isn’t exactly 0, rather than trusting a reporter’s pass/fail framing to never have an edge case. That’s the same “trust but verify the gate itself” instinct validate_response.py --selftest applies one layer down, applied here to the harness checking the deployed system.

Load testing: bounded, not benchmark

The pyramid’s thinnest layer, by design, is load. verify.sh’s Step 5 runs the collection’s Load folder (a single POST /bookings request) thirty times against the Quarkus stack with newman -n 30, and the assertion it checks is the same kind of assertion every layer below it has already been making: every one of the thirty iterations returns 201 with a schema-conforming body. Nowhere does this step assert a latency number, a percentile, or a throughput target, and that omission is the entire point. A CI gate that asserts “p95 latency under 200ms” is a gate that fails on a noisy shared runner having a slow morning, with nothing wrong in the code that triggered the failure — exactly the kind of flakiness that trains a team to re-run CI until it goes green instead of trusting what red means. Asserting success and schema conformance across repetition catches a real class of bug (a shared in-memory counter that corrupts under concurrent writes, a connection pool that exhausts on the fourth or fifth call and starts timing out) without asking a CI runner’s timing to be reproducible, which it structurally isn’t.

That’s also exactly why this layer stays this small. Thirty iterations against one stack is not a load test in the sense a capacity-planning exercise means it — it’s a teaching harness proving the pyramid has a layer above end-to-end at all, bounded tightly enough that it adds seconds to a CI run rather than minutes and never introduces timing-dependent flakiness into a gate other chapters’ work depends on passing. A team that needs to know how a service behaves under sustained concurrent load, with ramping request rates, connection reuse patterns, and real percentile reporting, reaches for a dedicated tool built for that question (k6, Gatling, Locust) run on its own schedule, against its own environment, answering “how does this system degrade under pressure” rather than “did these thirty requests all succeed,” which is a different question this chapter’s harness was never built to answer.

Figure 24.1 — The test pyramid for a REST API: contract, integration, end-to-end, and a thin bounded load layer, split by what runs in CI and what runs locally.

Download .excalidraw source Edit on excalidraw.com

Decision

Build as much of the pyramid as the API's consequences of being wrong justify, not the whole thing reflexively for every service. A contract test is close to free once a shared schema already exists (Chapter 4 already assumed one), so there's rarely a reason to skip it; it's the cheapest layer in this chapter and the one every other layer in this chapter builds on. Integration tests are worth adding the moment business logic exists beyond passing a request straight through to a datastore (a quote formula, a state-transition rule, an error-shaping decision), and belong at build time and in a developer's own loop, not in the CI gate everyone else waits on, the same way `@QuarkusTest` and `pytest` stay out of `verify.sh` here while remaining real and runnable. End-to-end tests belong in CI specifically when more than one deployable artifact needs to agree on one contract (two independently implemented stacks sharing a schema, the way this chapter's Quarkus and FastAPI services do), because that's the condition an in-process integration test structurally cannot check; a single-implementation API can often get by on contract and integration tests alone, reaching for Newman-style E2E only once a second implementation, a second team, or a public-facing client enters the picture. A thin, bounded load layer belongs in CI as a correctness check under light repetition, never as a latency gate — the moment anyone needs an actual answer about sustained throughput, concurrency limits, or degradation under pressure, that's the signal to stand up a dedicated load tool on its own infrastructure and its own schedule, not to keep stretching a thirty-iteration Newman run past the question it was built to answer.