PRISMPRISM
Evaluation pipeline
Validator pipeline from intake through schema checks, sandbox execution, benchmark scoring, consensus, and publication.
prismevaluationvalidatorspipeline
Sources
Stages
Every stage should emit structured status so miners can distinguish submission errors from infrastructure failures.
| Stage | Output |
|---|---|
| Intake | Submission ID, hotkey, manifest hash, artifact inventory. |
| Static validation | Schema, checksum, policy, dependency results. |
| Build | Image digest, install logs, model construction report. |
| Smoke test | Forward pass, memory estimate, observed metadata. |
| Benchmark | Per-task metrics, logs, resource usage. |
| Consensus | Validator agreement and outlier handling. |
| Publication | Leaderboard row and reward update. |
Validator assignments
Reward-bearing submissions should be assigned to multiple validators and escalated when they change top ranks or ownership status.