There is one answer an assessment office should never have to give: "the AI decided." When a program asks why its review came out as it did, it deserves a reply from a person who read the plan, weighed the evidence and stands behind the result. ProverbAI is designed to make that person better informed and faster, not to take their place.
This note describes how sign-off works. Reviewer sign-off is still in development within our private testing, so read what follows as a description of how the first release will work when it reaches institutions.
The order of work
- A program submits its annual assessment plan.
- ProverbAI reviews the plan. Every finding is backed by evidence from the plan, and where the plan does not address something the review expects, the finding says so.
- ProverbAI drafts feedback for the program.
- The assessment office assigns the review to a reviewer, or takes it on directly.
- The reviewer signs off. Until they do, the result stays inside the assessment office.
Reports are produced only when every plan that came in has been signed off, and only then are they released to programs. A plan that arrives after the deadline gets its own report later.
What a reviewer can do
The reviewer sees the plan alongside ProverbAI's review, with the evidence for each finding in view. From there, they have four options.
Confirm
If each finding holds up against the passage it cites and the feedback is fair, the reviewer confirms the result. Confirmation is a decision, recorded with the reviewer's name and the date. It is not a formality.
Edit the feedback
The drafted feedback is a starting point. A reviewer who knows the program, or who sees a clearer way to say something, can rewrite it. The program head reads the approved version, never the draft.
Adjust a finding, with a reason
If the reviewer reads the cited evidence and disagrees with a finding, they can change it, but only with a written reason. The program's result then reflects the change.
Review independently and compare
For any plan, a reviewer can set ProverbAI's work aside, carry out a full review of their own and compare the two. It takes the longest, and it is the right choice when a plan is unusual or a result looks wrong.
Why every change needs a reason
A change without a reason is one opinion quietly replacing another. With a reason, it becomes part of the record. Next year's reviewer can see what changed and why. The assessment office can see where reviewers and ProverbAI tend to differ, and whether the same issue keeps recurring. And when a program head asks why a finding differs from ProverbAI's review, the office has an answer to give.
Why results are not overwritten by hand
Reviewers adjust findings, not final results. A program's result follows from its findings, and the findings rest on the plan. If a person could simply replace a result, the line from that result back to the plan's own words would break, and comparisons across the college and the institution would mix two different kinds of judgement. Keeping results tied to findings means every outcome can still be explained, even after a reviewer has changed something.
Every plan, reviewed by a person
In the first year, a reviewer confirms every plan, whatever ProverbAI found. This is not sampling or spot-checking. ProverbAI's part is to make each review a matter of weighing evidence rather than searching for it, so that reviewers spend their time on judgement and feedback.
We are evaluating ProverbAI against trained reviewers on real plans, and will publish results once the evidence is sufficient. Until then, sign-off is not a safeguard added for appearances. It is how the product works.
