Skip to content

Result classifier

from fastmcp_feedback.instrumentation import DEFAULT_RESULT_CLASSIFIER, soft_error_classifier

A result classifier is (tool: str, result: Any) -> str | None.

  • It runs only for calls that did not raise.
  • result is the payload the record stores: the tool’s structured content when present, otherwise its content items (texts, or {"type": ...} for other kinds).
  • When the tool marked its result as an MCP error (isError) without raising, result arrives as a dict with "isError": True, alongside the payload’s own keys or with the payload under "content".
  • None means ok. A string marks a soft error: outcome="soft_error", ok=False, error_type="SoftError", and the string, redacted and cut to max_error_chars, becomes error_message.
  • If it raises, the error is logged and the call is recorded as ok.
  • It never changes what the client receives.
soft_error_classifier(*, statuses=("error", "failed", "failure"),
status_keys=("status", "state"),
flag_keys=("ok", "success"),
extra_statuses=(),
error_keys=("error",),
message_keys=("error", "message", "detail", "reason"),
) -> ResultClassifier
ParameterDescription
statusesValues that mean failure at a status key, compared case-insensitively.
status_keysKeys holding a status.
flag_keysKeys holding a success boolean.
extra_statusesAdded to statuses.
error_keysKeys whose non-empty string value marks a failure on its own. () turns the check off. Dotted paths allowed.
message_keysWhere <detail> comes from, first non-empty wins. Dotted paths allowed.

Non-dict results are always ok. For a dict, checks run in this order and the first that applies decides:

OrderConditionResultMessage
1a status key holds a failing statussoft errorstatus=failed: <detail>
2a flag key is exactly Falsesoft errorok=False: <detail>
3isError or is_error is Truesoft errorisError: <detail or content text>
4a flag key is exactly Trueok
5an error key holds a non-blank stringsoft errorerror: <string>
6none of the aboveok

<detail> is the first non-empty value at message_keys (error, message, detail and reason by default); a dict there is searched one level down for the same keys. Lists are skipped. A label without a detail is used alone, such as status=failed. For an error-key hit the label is the key as written, path included (result.error: <string>).

message_keys and error_keys entries can be paths into nested dicts: "snapshot.error" reads result["snapshot"]["error"]. The rules:

  • A key that literally contains a dot is tried first, so a result with a "snapshot.error" key of its own uses that.
  • Otherwise the path is split on every dot and walked. A missing step, or a step that is not a dict (None, a string, a list), means the key is not there. A lookup never raises.
  • status_keys and flag_keys are plain keys.

DEFAULT_RESULT_CLASSIFIER is soft_error_classifier() with these defaults.

classify = soft_error_classifier(extra_statuses=("unknown_target",))
assert classify("render", {"status": "unknown_target", "error": "no such object"}) == "status=unknown_target: no such object"
assert classify("render", {"status": "failed", "error": {"code": 7, "message": "disk full"}}) == "status=failed: disk full"
assert classify("render", {"error": {"code": 1}}) is None
assert DEFAULT_RESULT_CLASSIFIER("t", {"isError": True, "content": ["boom"]}) == "isError: boom"
nested = soft_error_classifier(error_keys=("result.error",), message_keys=("detail", "error"))
assert nested("t", {"result": {"error": "broke"}}) == "result.error: broke"
assert nested("t", {"result": "not a dict"}) is None
assert nested("t", {"status": "failed", "error": "e", "detail": "d"}) == "status=failed: d"
ReleaseChange
2026.09.27.4Classification added; outcome column.
2026.10.01error_keys check, on by default.
2026.10.01.1An explicit True flag wins over an error string.
2026.10.04message_keys; dotted paths in message_keys and error_keys.