Skip to main content

Run Tags

Each evaluation run can carry a set of free-form tags — short string labels that organise runs by experiment, environment, branch, or any other dimension that matters to you. Tags surface in the UI and can be filtered on, making them the natural way to group runs that belong together.

Common tagging conventions:

  • A commit or branch identifier — commit:abc123, branch:feature-rerank.
  • An environment — production, staging, local.
  • An experiment name — experiment-bm25-tuning, ranker-v2.
  • A release — release-2026.05.

Setting tags when creating a run​

Pass a tags array in the Create Evaluation Run request body:

curl -X POST "https://${RELEVAL_HOST}/api/v1/evaluations/${EVALUATION_ID}/runs" \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "name": "Re-ranker A/B run",
    "scale": "graded",
    "metrics": ["NDCG@10", "MAP"],
    "tags": ["commit:abc123", "experiment-bm25-tuning", "staging"]
  }'

In the UI, tags can be entered when creating the run alongside name and metrics.

Updating tags on an existing run​

Tags can be changed at any point in a run's lifecycle, including after it has completed or been locked. Use the Update Evaluation Run Tags endpoint:

curl -X PUT "https://${RELEVAL_HOST}/api/v1/evaluations/runs/${RUN_ID}/tags" \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "tags": ["commit:abc123", "release-2026.05"]
  }'

The request replaces the run's tag set with the supplied list. To clear all tags, send an empty array:

{ "tags": [] }

Limits​

  • A run can carry up to 50 tags.
  • Each tag must be non-empty and at most 200 characters.
  • Tags are case-sensitive and stored as-is; pick a convention and stick to it.

Using tags in CI​

Tagging runs with the commit SHA from your CI pipeline gives you a direct mapping from a commit to its evaluation result, which is useful when reviewing regressions:

COMMIT_SHA=$(git rev-parse --short HEAD)

curl -X POST "https://${RELEVAL_HOST}/api/v1/evaluations/${EVALUATION_ID}/runs" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d "{
    \"name\": \"CI run ${COMMIT_SHA}\",
    \"scale\": \"graded\",
    \"metrics\": [\"NDCG@10\", \"MAP\"],
    \"tags\": [\"commit:${COMMIT_SHA}\", \"branch:${CI_BRANCH}\", \"ci\"]
  }"

See App Clients for a complete CI integration example, including authentication.