Create an eval case
Adds one case to the suite. Both title and a non-empty steps array are required — steps is optional on the shared case shape so PATCH can be partial, but a case persisted without executable steps could never run. When models is omitted the suite’s configured model is used.
Authorizations
MCPJam API key (sk_…). Create one at Settings → API keys. Guest sessions cannot use the API, and API keys cannot manage other API keys.
Path Parameters
ID of the hosted project that contains the server.
Eval suite ID, as returned by POST /eval-runs.
Body
Create one case. title and a non-empty steps array are both required — a case persisted without executable steps could never run.
1REPLACES the case's test definition wholesale when provided.
1Stable declared id for the case — the id it answers to in a suite file, an import or a CLI argument. Minted for you when omitted, and unique within the suite. Distinct from the id on the response, which is the case's platform row id.
^[A-Za-z0-9_-]{1,128}$1 <= x <= 10Per-case repetitions under verdict policy 2; omitted cases inherit the suite default.
1 <= x <= 100Per-case pass threshold fraction under verdict policy 2.
0 <= x <= 1Optional authored analytics grouping label. Must be already trimmed.
1 - 64^\S(?:[\s\S]*\S)?$null clears the case override and inherits the suite's.
null clears the case override.
What a converter CLAIMED about one imported case. exact is CONVERTER-CLAIMED exact — the converter says it applied a structural mapping rule, cited in note. MCPJam has NOT verified semantic equivalence, so user-facing copy must say "claimed exact", never "verified" or "accepted". Claim-only: who approved an approximation, when, and why is a PER-RUN decision frozen on the run (ImportEligibility.approvedApproximationReceipts), never stored on the case — an approval that lived on a case would outlive the run it was granted for and the edit that invalidated it. Approval and internal keys are rejected with 400, never stripped.
Response
The created case.
A persisted eval case, in the public steps-first shape. Note this is NOT EvalTestCase, which is the INLINE authoring shape accepted by suite creation.
Ordered test steps. A prompt step is a model turn; a single model-free toolCall step is a render-check; assert steps hold the expectations.
11 <= x <= 10When true, the case passes if NO tools are called.
The case's effective declared id. Absent on cases authored before declared identity existed.
Trials this case runs under verdict policy 2, overriding the suite default. Absent means the case inherits it. NOT a second spelling of iterations: that one is the legacy count, which the legacy resolver reads as a FLOOR (max(iterations, suite.minimumIterations)) and which a policy-2 case still reports for compatibility. This one is exact.
x >= 1Fraction of this case's trials that must pass, overriding the suite default. Absent means the case inherits it. Never derived from the suite's minimumAccuracy, which is a PERCENT under a different resolver.
0 <= x <= 1Optional authored analytics grouping label. Must be already trimmed; absent means unlabelled.
1 - 64^\S(?:[\s\S]*\S)?$Absent when the case sets none — omitted from the response rather than sent as null.
Absent when the case sets none — omitted from the response rather than sent as null.
What a converter CLAIMED about one imported case. exact is CONVERTER-CLAIMED exact — the converter says it applied a structural mapping rule, cited in note. MCPJam has NOT verified semantic equivalence, so user-facing copy must say "claimed exact", never "verified" or "accepted". Claim-only: who approved an approximation, when, and why is a PER-RUN decision frozen on the run (ImportEligibility.approvedApproximationReceipts), never stored on the case — an approval that lived on a case would outlive the run it was granted for and the edit that invalidated it. Approval and internal keys are rejected with 400, never stripped.

