Skip to main content
POST
/
v2
/
evaluators
/
agent-tool-selection-quality
/
execute
Execute agent-tool-selection-quality evaluator
curl --request POST \
  --url https://api.traceloop.com/v2/evaluators/agent-tool-selection-quality/execute \
  --header 'Authorization: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "input": {
    "input_messages": "[{\"role\":\"system\",\"content\":\"You are a helpful assistant with access to tools: 1. get_weather(city) - Get current weather. 2. search_web(query) - Search the web.\"},{\"role\":\"user\",\"content\":\"What is the weather like in Paris?\"}]",
    "output_messages": "[{\"role\":\"assistant\",\"tool_calls\":[{\"name\":\"get_weather\",\"arguments\":{\"city\":\"Paris\"}}]}]"
  }
}
'
{
  "correct_args": true,
  "correct_args_reason": "The city argument matches the user's request",
  "tool_selection": "correct",
  "tool_selection_reason": "The agent correctly chose get_weather for the weather query"
}

Authorizations

Authorization
string
header
required

Type "Bearer" followed by a space and JWT token.

Body

application/json

Request body

input
object
required

Response

OK

correct_args
boolean
Example:

true

correct_args_reason
string
Example:

"The city argument matches the user's request"

tool_selection
string
Example:

"correct"

tool_selection_reason
string
Example:

"The agent correctly chose get_weather for the weather query"