Test-run settings API

Every project carries twelve test-run settings that govern how marking behaves during execution — whether a closed run still accepts result writes, whether an assignee lock applies, what happens after a result is recorded, and more. Eleven of the twelve inherit through a three-tier chain, project → organization → code default; the twelfth, defaultAssigneeUserId, is project-only and does not inherit at all.

{projectId} is the project code (for example ACME) everywhere below; {orgUlid} is the caller’s organization ULID.

Vocabulary and code defaults

Key	Type	Code default	Inherits?
autoCompleteRun	boolean	true	Yes
allowResultsInClosedRuns	boolean	true	Yes
fastPass	boolean	true	Yes
defectToggleDefault	boolean	true	Yes
autoAssignOnOpen	boolean	true	Yes
failCaseOnStepFail	boolean	false	Yes
autoPassWhenAllStepsPass	boolean	false	Yes
assigneeResultLock	boolean	false	Yes
requireCommentOnNegativeResult	boolean	false	Yes
afterAddingResult	"stay" | "next_pending_by_number" | "next_pending_visual"	"next_pending_visual"	Yes
timeTracking	"off" | "optional" | "required"	"optional"	Yes
defaultAssigneeUserId	member ULID | null	null	No — project-only

A project and organization that store no value for any key behave exactly as the product did before this capability existed — every code default above reproduces today’s observable behavior, with one deliberate exception: allowResultsInClosedRuns defaults to true, which formalizes a freeze-relaxation behavior already live on the single-mark path before this capability shipped. An organization that wants the older, stricter behavior sets it to false.

What each setting governs

• autoCompleteRun — whether a run closes itself automatically once every case has a result. When false, a run stays open until a member explicitly closes it.
• allowResultsInClosedRuns — whether a closed, non-aborted run still accepts result writes (marks, step marks, result-notes edits, result attachments). An aborted run is frozen under every setting value, always. See Run cases for the exact 409 behavior.
• fastPass / defectToggleDefault / requireCommentOnNegativeResult — web-client-only execution UX: whether marking passed/skipped bypasses the result modal, the modal’s defect-toggle seed, and whether a negative result requires a comment before submitting. See Known limitation below for the one setting in this group with no server-side counterpart.
• autoAssignOnOpen — whether opening an unassigned case auto-assigns the acting user.
• failCaseOnStepFail / autoPassWhenAllStepsPass — whether a step-level outcome derives the parent case’s status automatically. Precedence when both apply: failed step wins over blocked step wins over an all-passed step set.
• assigneeResultLock — whether only the assigned tester (or an exempt caller) may record a result for an assigned case. See Run cases for the 403 behavior.
• afterAddingResult — which case the execution board focuses next after a mark, in the web client only. Not observable over the API.
• timeTracking — whether elapsedMs is optional or required on a terminal single-case mark. See Run cases for the 422 behavior.
• defaultAssigneeUserId — pre-fills a new run’s default assignee. See Run-creation precedence below.

Resolution chain and source

Each of the eleven inheritable keys resolves in this order: an explicit project-level override wins; otherwise an explicit organization-level default wins; otherwise the code default applies. A stored null at either tier means “no override at that tier” — it is never treated as the value false or an empty string.

The response for each inheritable key carries:

Field	Meaning
value	The effective, resolved value — what actually governs behavior right now
source	"project", "organization", or "default" — which tier produced value
projectValue	The raw project-level stored override, or null
organizationValue	The raw organization-level stored override, or null
organizationEffective	The organization’s own effective value (its override, or the code default)

source is what a client should switch on to render an “overridden” / “inherited from organization” / “default” indicator — never compare value against organizationEffective. A project value that happens to equal the organization’s effective value is still a stored override: it must keep showing as one, and stay clearable, or a later organization default change would silently start affecting a project the user believed was pinned.

defaultAssigneeUserId is chain-exempt — it has no organization tier at all, so its entry carries only value and source ("project" or "default"), never projectValue, organizationValue, or organizationEffective.

Project settings

Get project settings

GET/api/v1/projects/{projectId}/run-settings

Open to any project reader (viewer+). Returns 200 with all twelve entries.

{
  "settings": {
    "autoCompleteRun": {
      "value": true,
      "source": "default",
      "projectValue": null,
      "organizationValue": null,
      "organizationEffective": true
    },
    "fastPass": {
      "value": false,
      "source": "project",
      "projectValue": false,
      "organizationValue": null,
      "organizationEffective": true
    },
    "timeTracking": {
      "value": "required",
      "source": "organization",
      "projectValue": null,
      "organizationValue": "required",
      "organizationEffective": "required"
    },
    "defaultAssigneeUserId": {
      "value": "01J...MEMBER",
      "source": "project"
    }
  }
}

(Abbreviated — the real response carries all twelve keys.)

Update project settings

PATCH/api/v1/projects/{projectId}/run-settings

Every key is optional — an absent key is left unchanged, and an explicit null resets that key back to inheriting (for defaultAssigneeUserId, back to no pre-fill). At least one key is required; an empty body is rejected with 422 validation_failed. Unknown keys are rejected the same way (.strict()).

{ "fastPass": false, "requireCommentOnNegativeResult": null }

defaultAssigneeUserId accepts a member’s ULID, never the internal integer id, resolved against the project’s organization’s active membership — an unknown or foreign ULID is rejected with 404 not_found, and nothing is persisted.

Returns 200 with the full, freshly resolved response (same shape as GET).

Authorization

Requires the projects.manage permission (held by owner/admin), or the project’s current owner. A member who can only read the project (viewer/member) receives 403 forbidden.

Example

curl -sS -X PATCH \
  -H "Authorization: Bearer $PROBARA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"fastPass": false, "assigneeResultLock": true}' \
  "https://app.probara.net/api/v1/projects/ACME/run-settings"

Organization defaults

Get organization defaults

GET/api/v1/orgs/{orgUlid}/run-settings

Open to any organization member. Returns 200 with the eleven inheritable keys only — defaultAssigneeUserId never appears here, since it has no organization tier.

{
  "settings": {
    "autoCompleteRun": { "value": true, "source": "default", "organizationValue": null },
    "fastPass": { "value": true, "source": "organization", "organizationValue": true }
  }
}

(Abbreviated — the real response carries all eleven keys.)

Update organization defaults

PATCH/api/v1/orgs/{orgUlid}/run-settings

Same absent-unchanged / explicit-null-resets-to-code-default semantics as the project endpoint, restricted to the eleven inheritable keys. Submitting defaultAssigneeUserId here is rejected with 422 validation_failed as an unknown key — never a silent no-op, since the schema only declares the eleven inheritable keys.

{ "timeTracking": "required" }

Authorization

Requires the org-settings.manage permission.

Example

curl -sS -X PATCH \
  -H "Authorization: Bearer $PROBARA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"autoCompleteRun": false}' \
  "https://app.probara.net/api/v1/orgs/01J...ORG/run-settings"

”Set as organization default”

Changing an organization default moves the effective value of every project that stores no override of its own for that key — a project already carrying its own project-sourced override is unaffected. The web app’s project settings panel offers a “Set as organization default” action in the row menu of any project-overridden key (visible only to a caller with org-settings.manage), which is exactly a PATCH to the organization endpoint with that one key’s project value — there is no separate API surface for it.

Known limitation: requireCommentOnNegativeResult is client-enforced

This one setting has no server-side counterpart. The comment text arrives on a separate request from the mark itself (PATCH /api/v1/runs/{runUlid}/results/{resultUlid}), so no single server request can observe “this case was marked failed with no comment” — enforcing it server-side would mean either widening the mark request with a breaking change disproportionate to one UI setting, or a delayed sweep with no natural trigger. An API or reporter client can mark a case failed or blocked with no comment while this setting is on — the web app’s result modal is the only place this rule is enforced. Every other setting in this document either inherits into server-side behavior or has an explicit, separately documented gap (see Run cases for timeTracking’s server-side enforcement, which is real).

Where timeTracking: "required" does and does not apply

timeTracking: "required"’s 422 elapsed_ms_required rejection (see Run cases) only fires on a code path that actually calls the single-case mark logic. This table is the complete surface audit:

Surface	Reaches the guard?	Enforces or exempt
Single-case mark — PATCH /api/v1/runs/{runUlid}/cases/{runCaseUlid}	Yes, directly	Enforces the 422
MCP mark_run_case tool	Yes — dispatches to the same route verbatim	Enforces — no special-casing in the tool, it inherits the route’s 422
Bulk mark — POST .../cases/bulk-mark (and MCP bulk_mark_run_cases)	No — the request schema carries no per-case duration field at all	Exempt at the schema level — there is no way to construct a violating request
Bulk submit-result — POST .../cases/bulk-submit-result (and MCP bulk_submit_result)	No — routes through a different service, never the single-mark path	Exempt by design — bulk submission is explicitly out of scope for this rejection
Result-notes enrichment — PATCH /api/v1/runs/{runUlid}/results/{resultUlid}	No — patches an already-appended result row’s notes; never mints a status change	Exempt — structurally out of scope, it carries no status
Automated result-ingestion route	N/A — this API has no CSV/webhook/CI result-ingestion surface today	Not applicable

Run-creation precedence

defaultAssigneeUserId only decides what a newly created run starts with — changing the project setting never rewrites an existing run’s stored assignee. POST /api/v1/projects/{projectId}/runs resolves the run’s default assignee in this order:

1. defaultAssigneeUlid present in the create-run request body (including an explicit null, meaning “deliberately none”) — always wins.
2. Otherwise, the project’s defaultAssigneeUserId setting, when set.
3. Otherwise, null — today’s behavior, unchanged.

Related

• Run cases — the marking endpoints these settings govern, including the 403 run_case_assignee_locked and 422 elapsed_ms_required behaviors
• Run result attachments — closed-run write behavior for step attachments and result notes
• API reference overview — pagination, errors, authentication
• Interactive v1 reference — full schemas