# Test-run settings API

> Read and change the twelve test-run settings that govern marking, closed-run writes, and run-creation defaults, at the project and organization level.

Every project carries twelve **test-run settings** that govern how marking behaves during
execution — whether a closed run still accepts result writes, whether an assignee lock applies,
what happens after a result is recorded, and more. Eleven of the twelve **inherit** through a
three-tier chain, `project → organization → code default`; the twelfth,
`defaultAssigneeUserId`, is **project-only** and does not inherit at all.

`{projectId}` is the project **code** (for example `ACME`) everywhere below; `{orgUlid}` is the
caller's organization ULID.

## Vocabulary and code defaults

| Key                              | Type                                                          | Code default            | Inherits?             |
| -------------------------------- | ------------------------------------------------------------- | ----------------------- | --------------------- |
| `autoCompleteRun`                | `boolean`                                                     | `true`                  | Yes                   |
| `allowResultsInClosedRuns`       | `boolean`                                                     | `true`                  | Yes                   |
| `fastPass`                       | `boolean`                                                     | `true`                  | Yes                   |
| `defectToggleDefault`            | `boolean`                                                     | `true`                  | Yes                   |
| `autoAssignOnOpen`               | `boolean`                                                     | `true`                  | Yes                   |
| `failCaseOnStepFail`             | `boolean`                                                     | `false`                 | Yes                   |
| `autoPassWhenAllStepsPass`       | `boolean`                                                     | `false`                 | Yes                   |
| `assigneeResultLock`             | `boolean`                                                     | `false`                 | Yes                   |
| `requireCommentOnNegativeResult` | `boolean`                                                     | `false`                 | Yes                   |
| `afterAddingResult`              | `"stay" \| "next_pending_by_number" \| "next_pending_visual"` | `"next_pending_visual"` | Yes                   |
| `timeTracking`                   | `"off" \| "optional" \| "required"`                           | `"optional"`            | Yes                   |
| `defaultAssigneeUserId`          | member ULID `\| null`                                         | `null`                  | **No — project-only** |

A project and organization that store no value for any key behave exactly as the product did
before this capability existed — every code default above reproduces today's observable
behavior, with one deliberate exception: `allowResultsInClosedRuns` defaults to `true`, which
formalizes a freeze-relaxation behavior already live on the single-mark path before this
capability shipped. An organization that wants the older, stricter behavior sets it to `false`.

### What each setting governs

- **`autoCompleteRun`** — whether a run closes itself automatically once every case has a
  result. When `false`, a run stays open until a member explicitly closes it.
- **`allowResultsInClosedRuns`** — whether a **closed, non-aborted** run still accepts result
  writes (marks, step marks, result-notes edits, result attachments). An **aborted** run is
  frozen under every setting value, always. See [Run cases](https://docs.probara.net/en/developers/api/run-cases/) for
  the exact `409` behavior.
- **`fastPass` / `defectToggleDefault` / `requireCommentOnNegativeResult`** — web-client-only
  execution UX: whether marking `passed`/`skipped` bypasses the result modal, the modal's
  defect-toggle seed, and whether a negative result requires a comment before submitting. See
  [Known limitation](#known-limitation-requirecommentonnegativeresult-is-client-enforced) below
  for the one setting in this group with no server-side counterpart.
- **`autoAssignOnOpen`** — whether opening an unassigned case auto-assigns the acting user.
- **`failCaseOnStepFail` / `autoPassWhenAllStepsPass`** — whether a step-level outcome derives
  the parent case's status automatically. Precedence when both apply: failed step wins over
  blocked step wins over an all-passed step set.
- **`assigneeResultLock`** — whether only the assigned tester (or an exempt caller) may record a
  result for an assigned case. See [Run cases](https://docs.probara.net/en/developers/api/run-cases/) for the `403`
  behavior.
- **`afterAddingResult`** — which case the execution board focuses next after a mark, in the web
  client only. Not observable over the API.
- **`timeTracking`** — whether `elapsedMs` is optional or required on a terminal single-case
  mark. See [Run cases](https://docs.probara.net/en/developers/api/run-cases/) for the `422` behavior.
- **`defaultAssigneeUserId`** — pre-fills a new run's default assignee. See
  [Run-creation precedence](#run-creation-precedence) below.

## Resolution chain and `source`

Each of the eleven inheritable keys resolves in this order: an explicit **project**-level
override wins; otherwise an explicit **organization**-level default wins; otherwise the **code
default** applies. A stored `null` at either tier means "no override at that tier" — it is never
treated as the value `false` or an empty string.

The response for each inheritable key carries:

| Field                   | Meaning                                                                     |
| ----------------------- | --------------------------------------------------------------------------- |
| `value`                 | The effective, resolved value — what actually governs behavior right now    |
| `source`                | `"project"`, `"organization"`, or `"default"` — which tier produced `value` |
| `projectValue`          | The raw project-level stored override, or `null`                            |
| `organizationValue`     | The raw organization-level stored override, or `null`                       |
| `organizationEffective` | The organization's own effective value (its override, or the code default)  |

`source` is what a client should switch on to render an "overridden" / "inherited from
organization" / "default" indicator — never compare `value` against `organizationEffective`.
A project value that happens to equal the organization's effective value is still a **stored
override**: it must keep showing as one, and stay clearable, or a later organization default
change would silently start affecting a project the user believed was pinned.

`defaultAssigneeUserId` is **chain-exempt** — it has no organization tier at all, so its entry
carries only `value` and `source` (`"project"` or `"default"`), never `projectValue`,
`organizationValue`, or `organizationEffective`.

## Project settings

### Get project settings

`GET /api/v1/projects/{projectId}/run-settings`

Open to any project reader (`viewer`+). Returns `200` with all twelve entries.

```json
{
  "settings": {
    "autoCompleteRun": {
      "value": true,
      "source": "default",
      "projectValue": null,
      "organizationValue": null,
      "organizationEffective": true
    },
    "fastPass": {
      "value": false,
      "source": "project",
      "projectValue": false,
      "organizationValue": null,
      "organizationEffective": true
    },
    "timeTracking": {
      "value": "required",
      "source": "organization",
      "projectValue": null,
      "organizationValue": "required",
      "organizationEffective": "required"
    },
    "defaultAssigneeUserId": {
      "value": "01J...MEMBER",
      "source": "project"
    }
  }
}
```

(Abbreviated — the real response carries all twelve keys.)

### Update project settings

`PATCH /api/v1/projects/{projectId}/run-settings`

Every key is **optional** — an **absent** key is left unchanged, and an explicit **`null`**
resets that key back to inheriting (for `defaultAssigneeUserId`, back to no pre-fill). At least
one key is required; an empty body is rejected with `422 validation_failed`. Unknown keys are
rejected the same way (`.strict()`).

```json
{ "fastPass": false, "requireCommentOnNegativeResult": null }
```

`defaultAssigneeUserId` accepts a member's **ULID**, never the internal integer id, resolved
against the project's organization's active membership — an unknown or foreign ULID is rejected
with `404 not_found`, and nothing is persisted.

Returns `200` with the full, freshly resolved response (same shape as `GET`).

#### Authorization

Requires the `projects.manage` permission (held by **`owner`**/**`admin`**), or the project's
**current owner**. A member who can only read the project (`viewer`/`member`) receives
`403 forbidden`.

#### Example

```bash
curl -sS -X PATCH \
  -H "Authorization: Bearer $PROBARA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"fastPass": false, "assigneeResultLock": true}' \
  "https://app.probara.net/api/v1/projects/ACME/run-settings"
```

## Organization defaults

### Get organization defaults

`GET /api/v1/orgs/{orgUlid}/run-settings`

Open to any organization member. Returns `200` with the **eleven** inheritable keys only —
`defaultAssigneeUserId` never appears here, since it has no organization tier.

```json
{
  "settings": {
    "autoCompleteRun": { "value": true, "source": "default", "organizationValue": null },
    "fastPass": { "value": true, "source": "organization", "organizationValue": true }
  }
}
```

(Abbreviated — the real response carries all eleven keys.)

### Update organization defaults

`PATCH /api/v1/orgs/{orgUlid}/run-settings`

Same absent-unchanged / explicit-`null`-resets-to-code-default semantics as the project endpoint,
restricted to the eleven inheritable keys. Submitting `defaultAssigneeUserId` here is rejected
with `422 validation_failed` as an **unknown key** — never a silent no-op, since the schema only
declares the eleven inheritable keys.

```json
{ "timeTracking": "required" }
```

#### Authorization

Requires the `org-settings.manage` permission.

#### Example

```bash
curl -sS -X PATCH \
  -H "Authorization: Bearer $PROBARA_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"autoCompleteRun": false}' \
  "https://app.probara.net/api/v1/orgs/01J...ORG/run-settings"
```

### "Set as organization default"

Changing an organization default moves the **effective** value of every project that stores no
override of its own for that key — a project already carrying its own `project`-sourced override
is unaffected. The web app's project settings panel offers a "Set as organization default" action
in the row menu of any project-overridden key (visible only to a caller with
`org-settings.manage`), which is exactly a `PATCH` to the organization endpoint with that one key's
project value — there is no separate API surface for it.

## Known limitation: `requireCommentOnNegativeResult` is client-enforced

This one setting has **no server-side counterpart**. The comment text arrives on a separate
request from the mark itself (`PATCH /api/v1/runs/{runUlid}/results/{resultUlid}`), so no single
server request can observe "this case was marked failed with no comment" — enforcing it
server-side would mean either widening the mark request with a breaking change disproportionate
to one UI setting, or a delayed sweep with no natural trigger. **An API or reporter client can
mark a case `failed` or `blocked` with no comment while this setting is on** — the web app's
result modal is the only place this rule is enforced. Every other setting in this document either
inherits into server-side behavior or has an explicit, separately documented gap (see
[Run cases](https://docs.probara.net/en/developers/api/run-cases/) for `timeTracking`'s server-side enforcement, which
_is_ real).

## Where `timeTracking: "required"` does and does not apply

`timeTracking: "required"`'s `422 elapsed_ms_required` rejection (see
[Run cases](https://docs.probara.net/en/developers/api/run-cases/)) only fires on a code path that actually calls the
single-case mark logic. This table is the complete surface audit:

| Surface                                                                                 | Reaches the guard?                                                                 | Enforces or exempt                                                                   |
| --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| Single-case mark — `PATCH /api/v1/runs/{runUlid}/cases/{runCaseUlid}`                   | Yes, directly                                                                      | **Enforces** the `422`                                                               |
| MCP `mark_run_case` tool                                                                | Yes — dispatches to the same route verbatim                                        | **Enforces** — no special-casing in the tool, it inherits the route's `422`          |
| Bulk mark — `POST .../cases/bulk-mark` (and MCP `bulk_mark_run_cases`)                  | No — the request schema carries no per-case duration field at all                  | **Exempt** at the schema level — there is no way to construct a violating request    |
| Bulk submit-result — `POST .../cases/bulk-submit-result` (and MCP `bulk_submit_result`) | No — routes through a different service, never the single-mark path                | **Exempt** by design — bulk submission is explicitly out of scope for this rejection |
| Result-notes enrichment — `PATCH /api/v1/runs/{runUlid}/results/{resultUlid}`           | No — patches an already-appended result row's `notes`; never mints a status change | **Exempt** — structurally out of scope, it carries no `status`                       |
| Automated result-ingestion route                                                        | N/A — this API has no CSV/webhook/CI result-ingestion surface today                | Not applicable                                                                       |

## Run-creation precedence

`defaultAssigneeUserId` only decides what a **newly created** run starts with — changing the
project setting never rewrites an existing run's stored assignee. `POST
/api/v1/projects/{projectId}/runs` resolves the run's default assignee in this order:

1. `defaultAssigneeUlid` **present** in the create-run request body (including an explicit
   `null`, meaning "deliberately none") — always wins.
2. Otherwise, the project's `defaultAssigneeUserId` setting, when set.
3. Otherwise, `null` — today's behavior, unchanged.

## Related

- [Run cases](https://docs.probara.net/en/developers/api/run-cases/) — the marking endpoints these settings govern,
  including the `403 run_case_assignee_locked` and `422 elapsed_ms_required` behaviors
- [Step actual results](https://docs.probara.net/en/developers/api/run-result-attachments/) — closed-run write
  behavior for step results and result notes
- [API reference overview](https://docs.probara.net/en/developers/api-reference/) — pagination, errors, authentication
- [Interactive v1 reference](https://docs.probara.net/en/developers/api-reference/v1/) — full schemas