Designing Stateful API Simulators so Your Tests Own Their Starting Conditions
Establish simulator state deliberately so tests don't depend on execution order.

Integration testing against stateful APIs requires a simulator that preserves changes across calls and lets each test declare its starting state without hitting production. Deliberate control over resources, event history, and active faults makes ordered workflows reproducible. Reset, seed, and snapshot give tests that control, while isolation keeps other tests from changing the state under test.
Why state setup is the part of simulator design most teams get wrong
A checkout flow creates a customer, creates a payment intent for that customer, confirms the intent, and then receives a webhook. Each step depends on what happened before it. A stateless mock can return a realistic success code for the first request without storing anything, leaving a follow-up GET for the same ID to return 404. It cannot preserve the customer relationship through the workflow or produce the webhook from that evolving state.
A stateful simulator retains state between requests, but memory alone is insufficient. The design gap is control over what that memory contains before a test begins. Without deliberate setup, one test’s writes pollute another test’s reads, reproducing the uncertainty of a shared staging environment. Results depend on execution order, parallel runs interfere with one another, and failures become difficult to reproduce.
What "owning your starting conditions" actually means in practice
A test owns its starting conditions when it can declare which resources exist, their current states, and the history the simulator believes has occurred before test logic runs. These are distinct parts of the same contract. A payment intent that exists in requires_confirmation is not interchangeable with one in succeeded, and a prior webhook changes the history even when the resource set is unchanged.
The practical criterion is a single setup call that clears existing state and restores a known scenario or snapshot. Repeated API calls to build prerequisites make the test depend on setup workflows as well as the integration being examined. Each worker must receive an independent state space, rather than share mutable resources that another worker can reset or change.
IcePick formalizes API state evolution with TLA+ and uses TLC to exhaustively explore reachable states, generating test sequences that provably cover the behavioral model. The guarantees from that coverage are limited to the behavioral model. It nevertheless sharpens the requirement for simulator design. Tests need a way to set the states they test without relying on earlier activity to leave the simulator in a useful state.
The three mechanisms that make state setup first-class: reset, seed, and snapshot
Reset returns the simulator to a known empty or factory baseline through a single endpoint or CLI call. It replaces teardown sequences whose delete calls can fail or leave orphaned resources. The reset must be atomic from the test’s perspective. A partial reset exposes an undefined intermediate state, while a complete reset establishes a usable boundary. Global reset clears everything and is easier to build, while scoped reset clears one test’s resources and can let tests run in parallel without separate processes.
Seed loads a cohesive starting dataset before execution. Empty state forces tests to construct prerequisites, adding noise and making setup failures harder to distinguish from integration failures. A mid-sized design studio scenario for Stripe, Slack, or PayPal instead gives tests records like those that would already be there. Within a payment scenario, customer records, payment intents, and invoices must cross-reference consistent IDs so that valid reads do not fail because the seed itself is incoherent.
Snapshot saves the simulator’s full state so it can be restored later. It makes mid-workflow conditions reusable without repeating their construction. A Stripe payment intent that is confirmed but not captured, or a GitHub PR with reviews but no merge, can become a named starting point. Labels such as “post-onboarding” or “pre-dispute” let tests share a baseline without depending on one another’s setup calls. Restoration must keep behavior exactly the same, including internal sequence counters and ID generators, so that assertions on specific IDs can still be trusted after restore.
How cohesive seed scenarios change what tests can assert
In a design studio scenario, customers, subscriptions, invoices, and payment history refer to one another through consistent IDs. That coherence expands what a test can ask. It can examine a customer’s outstanding balance, subscription status, or the last invoice’s line items without constructing those relationships in its body. Disconnected stubs can show that endpoints respond, but cannot support the same multi-resource assertions.
The same distinction applies to a GitHub integration. A large open-source repo scenario can include PRs, review threads, checks, and labels that support a review-and-merge sequence. A bare repository has none of that state to start from. The test must first build the resources that make the sequence meaningful, so its prerequisites compete with the behavior it is meant to examine. Cohesive seeds move that construction out of the test while retaining the relationships the integration depends on.
A Kalshi batch-order endpoint returning 429 is useful for examining an agent’s response to pressure when the simulator already holds open positions the agent is managing. An empty simulator cannot expose that interaction. The fault alone exercises a response path, while the surrounding state makes it possible to examine that path in the workflow where it is needed.
Isolating state between tests so parallelism doesn't produce noise
Parallel CI requires simultaneous ownership of starting conditions. If worker A resets a shared simulator and worker B then mutates it, worker A can read a state it never declared. The resulting failure reflects interference. Running that test alone removes the interference, which explains why the failure can disappear during investigation. For reset and seed to work reliably, their effects must stay within an isolation boundary.
Process-per-worker isolation gives each CI worker a simulator process with its own in-memory or on-disk state. It is the simplest model because workers have no shared simulator resources to coordinate, though each worker adds container overhead. Namespace or tenant isolation instead partitions state inside one process, making each partition invisible to the others. That design is harder to implement correctly. In particular, a scoped reset must respect the same partition boundary as the reads and writes it isolates.
For CI pipelines, the practical interface is a simulator that starts in a container, loads a seed from a file, and exposes reset over HTTP. The test runner can decide how many instances to launch, while a namespace-based deployment also needs the corresponding logical isolation. Reinforcement learning training asks more of that isolation because many environments must change without affecting one another. At that scale, the overhead of each process makes namespace isolation within a horizontally scalable simulator the practical choice, and each run still needs its own state.
Fault injection as a state-setup concern, not a separate testing layer
A retry test needs control over failure conditions just as a workflow test needs control over resources. Its setup must be able to declare that an endpoint will return 429 for the next N calls. That declaration should hold regardless of timing. Triggering faults by waiting for real rate limits replaces a known starting condition with uncertainty about when the failure will occur. Fault scheduling therefore belongs in the same interface that loads the seed.
Fault fidelity includes headers and status codes. Kalshi’s documentation states that its 429 responses include neither Retry-After nor X-RateLimit-* headers. Retry logic that assumes those headers will be present can silently fail, so a simulator that adds them would conceal the relevant behavior. The setup interface must allow their absence to be declared.
Other fault declarations need equally precise scope. A declared delay on a particular endpoint exercises timeout handling, while a truncated list payload exercises pagination and partial-response handling. An authentication failure can be an HTTP 400 with an invalid_grant error envelope at a specific endpoint. These conditions test different paths in the client. Their setup must distinguish a transient fault that clears after firing from one that persists until reset.
Trigger conditions complete that declaration by specifying when the fault applies. A simulator permanently returning 503 does not isolate the retry path of a particular workflow. By contrast, POST /v1/payment_intents returning 503 on the third call targets a defined point in that workflow. The endpoint or resource scope, trigger condition, and duration let the test say exactly which failure it expects.
Keeping the simulator's state model honest as the real API evolves
A reproducible state can still be an inaccurate one. If the simulator returns a field the provider has removed, or accepts a transition the provider now rejects, passing tests express confidence in the wrong model. Deliberate setup cannot correct that divergence by itself. Fidelity must be assessed as the real API evolves, rather than treated as a property established when the simulator was built.
Different comparisons reveal different forms of drift. Spec-to-spec diffing compares OpenAPI versions to detect field additions, removals, and type changes. Spec-to-reality monitoring compares live responses with the specification’s predictions, while reality-to-reality monitoring compares live responses with a learned behavioral baseline and requires no specification. The states and transitions the simulator accepts must still match the API that the integration will use.
That evolving model also changes the meaning of saved setup data. Seeds and snapshot formats must be versioned with the simulator because an older snapshot may become unrestorable or silently wrong under a newer model. Fidelity scores measured on an ongoing basis show how simulated and live behavior differ, giving engineers a basis for deciding how much to trust their tests. Verification before each release and publication of those scores make that basis explicit.
What a state-setup API should look like from the test's perspective
The test-facing contract should make the preceding capabilities available without setup scaffolding in the test body. Its reset operation, exposed through HTTP or CLI, should accept an optional seed name and return the simulator to that scenario in one call. A state inspection endpoint should show which resources are in the simulator after the test runs. Responses alone do not establish what was stored. Inspection lets assertions examine the simulator’s resulting state as well as the HTTP responses produced during the workflow.
The same setup call should accept a fault schedule with rules that name the endpoint, fault type, trigger condition, and duration. This keeps resource conditions and failure conditions in one declaration. Snapshot save and restore should use a test name or scenario label, allowing prepared state to be restored in milliseconds for reuse across runs. These operations must act within the test’s isolation boundary. Otherwise, a convenient interface would still permit one worker’s setup to change another worker’s execution.
Calls in the test should exercise the integration. If a function has API calls just to create starting state, the setup interface needs to give it more control. The setup interface separates the ordered calls that constitute the workflow under test from the customer, payment history, or review state that the workflow assumes already exists.
For CI, that separation should remain visible in a single pipeline step that loads the seed, configures faults, runs the test, inspects state, and resets. A readable sequence makes the failure point unambiguous. AI agent and reinforcement learning workflows need the same contract at environment initialization and each episode reset. Deterministic restoration then prevents prior episodes from contaminating the next training signal, preserving the same ownership of starting conditions that an integration test requires.


