Read this first
The invented business
An eight-person plumbing business in a regional town. Technicians type site notes into a job app. The office manager turns the notes into written quotes using a price-list spreadsheet. The owner wants to know whether an AI tool could draft the quotes.
Proposed use: a business-account AI assistant drafts the scope and line items of a quote from the site notes and the price list. The office manager checks and sends every quote.
Worksheet scores
| Criterion | Score | Evidence |
|---|---|---|
| How much time or money does this task cost you now? | 4 | Assumed for the scenario: the office manager spends most of one day a week turning technicians’ site notes into written quotes. |
| How often is the task done? | 5 | Several quotes are written every working day. |
| Is the information the AI would need digital, current and in one place? | 3 | Site notes are typed into the job app, but the price list is a separate spreadsheet that is not always current. |
| How quickly can a competent person check the AI’s output? | 4 | The office manager can check a draft against the notes and price list in a few minutes, and pricing errors are easy to see side by side. |
| If the output is wrong and nobody notices, what happens? | 3 | A wrong price or missing item leads to a dispute, a discount or rework. Nobody is harmed. |
| What information would go into the AI tool? | 3 | Designed out: customer names, addresses and phone numbers are not entered. The AI sees only the job notes; customer details are added from the job app afterwards. Pricing is internal (amber). |
Benefit 9/10 · Readiness 7/10 · Risk 6/10. Hold points triggered: none.
Decision: Go to a small pilot. Conditions:
- Write the acceptance criteria down before testing, and test on your own de-identified examples.
- A named person checks every output before it leaves the business.
- Use only an approved business account, not a personal or free account.
- Log anything that goes wrong as a non-conformance and fix the cause.
Note the design choice that kept sensitivity at 3: had customer details gone into the tool, sensitivity would have scored 4 and the “information rules needed first” hold point would have applied.
Pilot inspection and test plan
Agreed in writing before testing. The test set is 30 past jobs, with the quote the office manager actually sent as the reference answer.
| Check | Acceptance criterion | Method | Sample | By | Type |
|---|---|---|---|---|---|
| Prices | Every line-item price matches the current price list. No errors accepted. | Compare each line with the price list | All 30 test quotes | Office manager | Hold point |
| Scope | Every item in the site notes appears in the quote, and nothing appears that is not in the notes. | Side-by-side review | All 30 | Technician who wrote the notes | Witness point |
| Standard terms | The business’s standard exclusions and validity paragraph is present and unchanged. | Text comparison | All 30 | taliQ | Record |
| No personal information entered | No customer name, address or phone number in any prompt. | Review of every prompt input | All 30 | taliQ | Hold point |
| Time to check | Checking a draft takes less time than writing the quote from scratch. | Timed, with a stopwatch | 10 of the 30 | Office manager | Record |
Release rule: both hold points pass on all 30 quotes. If any check fails, an NCR is raised and the failing check is re-run after the fix.
Example non-conformance report
- NCR
- 001
- Check
- Scope
- What happened
- Test job 17: the notes said “HWS insp.” (hot water system inspection). The draft quote listed a hot water system replacement.
- Disposition
- Reject. The draft would not have been sent.
- Root cause
- Technicians’ abbreviations are not standard, and the prompt did not explain them.
- Corrective action
- Add the business’s abbreviation list to the prompt, and instruct the tool to flag any abbreviation it does not recognise instead of guessing.
- Verification
- Re-run the scope check on all 30 jobs, plus five extra jobs chosen for heavy abbreviation.
- Closed by
- Accountable owner, once verification passes
What the scenario shows
- The decision was made before any tool was bought, on evidence that is written down.
- A design choice (keeping customer details out) changed the risk score, and the reason is recorded.
- The test had pass marks agreed in advance, so a failure produced a fix and a re-test rather than an argument.
Try the same scoring on your own task with the use-case scorer.