Skip to main content

App Clients

App Clients provide machine-to-machine authentication for programmatic API access. They use the OAuth 2.0 client credentials flow to obtain JWT tokens, allowing scripts, CI/CD pipelines, and external applications to interact with Releval's API without user credentials.

Creating an App Client​

In the UI​

  1. Navigate to App Clients and click Create App Client
  2. Enter a Name and Description
  3. Pick one or more Roles for the client. You can only assign roles at or below your own highest role; see Members and Roles.
  4. Click Create App Client

You'll be presented with three values:

  • Authentication URL — the token endpoint for your deployment (e.g., https://${RELEVAL_HOST}/api/v1/oauth2/token)
  • Client ID — the unique identifier for this client
  • Client Secret — the secret used to authenticate
Info

The client secret is only shown once at creation time. Copy it immediately. If you lose the secret, you can regenerate a new one from the app client's detail page, which invalidates the previous secret.

Using the API​

curl -X POST "https://${RELEVAL_HOST}/api/v1/app-clients" \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer ${TOKEN}" \
-d '{
  "name": "CI Pipeline",
  "description": "Automated evaluation runs from CI",
  "roles": ["Member"]
}'

If you don't pick roles explicitly, the client inherits every role the calling member can assign — convenient for quick setup, but pick a tighter set in production. The same "can only assign roles at or below your own highest role" rule applies as for inviting members.

Authenticating with an App Client​

Step 1: Obtain a JWT Token​

Exchange the client credentials for a JWT token:

curl -X POST "https://${RELEVAL_HOST}/api/v1/oauth2/token" \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d "grant_type=client_credentials&client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}"

Response:

{
  "access_token": "eyJhbGciOiJIUzUxMiIs...",
  "token_type": "Bearer",
  "expires_in": 3600
}

Step 2: Use the Token​

Include the token in the Authorization header for API requests:

curl "https://${RELEVAL_HOST}/api/v1/evaluations" \
-H 'Authorization: Bearer eyJhbGciOiJIUzUxMiIs...'

Token Expiration​

Tokens expire after the configured JwtSettings__ExpireTimeSpan (default: 1 hour). When a token expires, request a new one using the same client credentials. See JWT Settings.

Managing App Clients​

List App Clients​

curl "https://${RELEVAL_HOST}/api/v1/app-clients" \
-H "Authorization: Bearer ${TOKEN}"

Regenerate Secret​

If a client secret is compromised, regenerate it:

curl -X POST "https://${RELEVAL_HOST}/api/v1/app-clients/${APP_CLIENT_ID}/regenerate-secret" \
-H "Authorization: Bearer ${TOKEN}"

This immediately invalidates the old secret and any cached tokens.

Delete App Clients​

curl -X DELETE "https://${RELEVAL_HOST}/api/v1/app-clients?app_client_id=${APP_CLIENT_ID}" \
-H "Authorization: Bearer ${TOKEN}"

Example: Automated Evaluation in CI​

A common use case is running evaluations automatically in a CI/CD pipeline:

#!/bin/bash

# 1. Authenticate
TOKEN=$(curl -s -X POST "https://${RELEVAL_HOST}/api/v1/oauth2/token" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials&client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}" \
  | jq -r '.access_token')

# 2. Create an evaluation run
RUN_ID=$(curl -s -X POST "https://${RELEVAL_HOST}/api/v1/evaluations/${EVALUATION_ID}/runs" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{"name": "CI Run '"$(date +%Y-%m-%d)"'", "scale": "graded", "metrics": ["NDCG@10", "MAP"]}' \
  | jq -r '.id')

# 3. Start the run
curl -s -X POST "https://${RELEVAL_HOST}/api/v1/evaluations/runs/${RUN_ID}/start" \
  -H "Authorization: Bearer ${TOKEN}"

echo "Evaluation run ${RUN_ID} started"