Skip to main content

Query Sets

Query sets are collections of queries that serve as input data for your search evaluations in Releval. They form a critical part of an evaluation workflow, providing the actual search terms that will be sent to your search endpoints.

Queries in a query set may represent a variety of different use cases:

  • A golden query set of carefully curated queries that represent real user intents.
  • A list or sensitive topics / problematic queries that need to be specifically handled.
  • Trending or popular queries from the last day or week.

Usage​

Query sets allow you to:

  • Evaluate search relevancy and performance across a representative sample of user queries
  • Test different query templates and therefore query construction, against consistent input
  • Compare search quality across multiple search configurations
  • Build reliable benchmarks for measuring improvements

Structure of a Query​

A query in a query set can be either

  • A text string of search terms e.g.

    The Godfather

or

  • A JSON object containing the search terms in a "query" property, and optionally other values e.g.

    { "query": "The Godfather", "genre": ["Crime", "Drama"] }
Info

All queries in a query set must either be strings or JSON objects, but not both; they cannot be intermixed within one query set.

When using JSON objects,

  • the property name "query" must be used for your search terms
  • properties with the same name must have the same type across JSON objects in a query set

"query" is the variable name used for search terms defined as strings in query templates. Using the same name in JSON objects keeps templates consistent.