Made-up example. Northbridge Bank is not a real bank, and every number here is invented to show how AI EnableOS works.
4
Step 4 of 6
Pilot
Build, test and run with a small group.
1. Questions asked
The AI wrote these questions from similar real projects. The answers are made up.
Q: Who will test it, and for how long?
A: 100 phone specialists at Site A, for 6 weeks.
Q: How will we know it worked?
A: Average call time, audit errors and weekly use, compared with a matching group.
Q: What are the numbers before the pilot?
A: 6.0 minutes per call, 4.0% audit errors, 96% correct answers, satisfaction 4.1 of 5.
Q: Did it pass testing?
A: 96% correct on 500 test questions, against a 95% bar.
2. What we found
- Pilot group
- 100 phone specialists at Site A
- Length
- 6 weeks
- Success measures
- Average call time, audit error rate, weekly use, against a matching comparison group
- Test results
- 96% correct answers on 500 test questions, against a 95% bar
- Numbers before the pilot (locked)
- 6.0 min per call, 4.0% audit errors, 96% correct answers, satisfaction 4.1 of 5
3. AI recommends, a person decides
AI recommends Go
Launch the pilot. It passed testing and the before-numbers are measured.
- 96% correct on 500 test questions, above the 95% bar.
- Before-numbers are measured and now locked, so results can't be moved later.
- A matching group without the tool gives a fair comparison.
A person decided: Model-risk team (sign-off 3)
4. Sign-off 3: Risk sign-off to launch
Who signs
Model-risk team
Evidence they looked at
- Test: 96% correct on 500 questions, bar is 95%
- Before-numbers measured and locked
- Comparison group named
Sent back to fix if
Test score is under the 95% bar, or the before-numbers are missing.
Stopped if
Wrong answers could harm customers and nobody would catch them.
Signed: Approved for 100 specialists, 6 weeks.