MCP Server

Releval ships with a built-in Model Context Protocol (MCP) server that exposes a curated subset of the API as tools an AI assistant can call. Point Claude Code or another MCP-compatible client at it and you can ask, in natural language, to "list my evaluations", "create a run for evaluation X", or "show me the judgments I've made for this query" — without writing curl commands by hand.

The MCP server lives at /mcp on the same host as your Releval REST API.

Authentication

The MCP server uses the same OAuth 2.0 client-credentials flow as the rest of the API. You authenticate by:

1. Creating an App Client for the MCP integration.
2. Exchanging the client credentials for a JWT bearer token at /api/v1/oauth2/token.
3. Sending the token in the Authorization: Bearer … header on MCP requests.

Most MCP clients (including Claude Code) handle the OAuth flow automatically once you tell them the server URL — they discover the authorisation server from the protected-resource metadata Releval exposes.

Available tools

Tools are grouped by domain. Each tool maps onto a REST endpoint with the same semantics — license limits, role checks, validation, and audit logging all apply to MCP calls just as they do to API calls.

Search endpoints

Tool	What it does
list_search_endpoints	Lists all configured search endpoints.
get_search_endpoint	Gets a single endpoint by ID.
create_search_endpoint	Creates a new endpoint and (optionally) attaches it to an existing corpus.
update_search_endpoint	Updates an endpoint's name, URL, method, or candidates mapping.
delete_search_endpoints	Deletes one or more endpoints.
test_search_endpoint	Tests an unsaved endpoint configuration with a live request.
test_existing_search_endpoint	Tests a saved endpoint, optionally overriding fields for the test only.

Query sets and templates

Tool	What it does
list_query_sets	Lists query sets, optionally including each set's queries.
get_query_set	Gets a query set with its queries.
create_query_set	Creates a new query set from inline queries.
delete_query_sets	Deletes one or more query sets.
list_query_templates	Lists all query templates.
get_query_template	Gets a single query template.
create_query_template	Creates a new Handlebars query template.
update_query_template	Updates an existing query template.
delete_query_templates	Deletes one or more query templates.
test_query_template	Renders a query template against sample queries.

Evaluations and runs

Tool	What it does
create_evaluation	Creates a new evaluation linking an endpoint, query set, and query template.
list_evaluations	Lists all evaluations.
get_evaluation	Gets a single evaluation.
update_evaluation	Updates an evaluation.
delete_evaluations	Deletes one or more evaluations.
create_evaluation_run	Creates a run for an evaluation, choosing a scale and metrics.
list_evaluation_runs	Lists runs for an evaluation, most recent first.
get_evaluation_run	Gets a run with status, metrics, and config.
start_evaluation_run	Starts a Pending run.
clone_evaluation_run	Clones a run with the same configuration.
lock_evaluation_run	Locks a run, freezing its judgments.
update_evaluation_run_tags	Sets tags on a run.
compare_evaluation_runs	Compares a run against a baseline.
delete_evaluation_runs	Deletes one or more runs.
list_evaluation_queries	Lists the queries on a run, paginated.
list_evaluation_query_results	Lists the candidates returned for a specific query.
list_metrics	Lists the available metrics that can be computed.

Judgments

Tool	What it does
list_judgments	Lists your judgments for a query within a corpus.
upsert_judgments	Creates or updates judgments.

Configuring an MCP client

Claude Code

Add Releval as an MCP server in Claude Code's settings. The exact UI depends on your version, but the configuration looks like:

{
  "mcpServers": {
    "releval": {
      "url": "https://${RELEVAL_HOST}/mcp"
    }
  }
}

Claude Code will prompt you to authenticate the first time it connects, and will perform the OAuth client-credentials exchange against /api/v1/oauth2/token using credentials you supply.

Generic MCP clients

Any MCP client supporting OAuth-protected resources can connect:

• Resource URL: https://${RELEVAL_HOST}/mcp
• Authorisation server: https://${RELEVAL_HOST}/api/v1/oauth2/token
• Grant type: client_credentials

The client should fetch protected-resource metadata at the resource URL to discover the authorisation server automatically.

Permissions

MCP calls inherit the role and license limits of the App Client whose token is presented:

• A client without Admin role cannot perform admin-only operations (e.g. creating shared resources on behalf of others).
• License limits apply identically — create_evaluation returns a license-limit error if the workspace is at its evaluation cap.
• All MCP tool invocations are logged with the calling member, the same as REST API calls.

When to use MCP vs. the REST API

Use MCP when an AI assistant is the natural driver — exploration ("what evaluations have I run this week?"), conversational construction ("create a run for the latest production endpoint, NDCG@10 and MAP, tag it experiment-rerank-v2"), or pulling Releval data into an LLM workflow.

Use the REST API for everything else: programmatic integration, CI pipelines, scheduled jobs, anything that benefits from explicit, replayable requests.