rai eval
Evaluation harness for retrieval quality. Runs benchmarks against the knowledge graph and pattern store to measure retrieval precision, recall, and ranking quality.
rai eval run¶
Run an evaluation suite against the current retrieval configuration. Outputs precision/recall metrics per query set.
| Flag | Short | Description |
|---|---|---|
--suite |
-s |
Evaluation suite name or path to a suite file |
--format |
-f |
Output format: human, json. Default: human |
--limit |
-n |
Maximum number of queries to evaluate |
# Run default suite
rai eval run
# Run a specific suite
rai eval run --suite qas-tier1
# JSON output for CI dashboards
rai eval run --format json
rai eval tune¶
Auto-tune retrieval parameters (weights, thresholds, strategy mix) against an evaluation suite to maximize a target metric.
| Flag | Short | Description |
|---|---|---|
--suite |
-s |
Evaluation suite to optimize against |
--metric |
Target metric to maximize: precision, recall, f1. Default: f1 |
|
--dry-run |
Print proposed parameter changes without applying them |
# Tune against default suite
rai eval tune
# Tune for recall on a specific suite
rai eval tune --suite qas-tier1 --metric recall
# Preview changes without applying
rai eval tune --dry-run
See also: rai graph, rai pattern