The Architecture Selection Compass: six questions before you upgrade the model
When does an AI workload need a verify-before-answer architecture, and when is single-shot fine? This is a decision tool, not an opinion. Run each workload through six axes and you get a defensible recommendation instead of a guess. It is the framework people asked for after we published the benchmark where verify-before-answer beat single-shot 8 of 10 on banking-compliance questions.
The context in one paragraph
Single-shot RAG retrieves once and answers. It is fast and cheap, and it fails by being confidently incomplete: in the benchmark, one answer scored 0.92 on relevance while missing two of three obligations. Verify-before-answer reformulates the question, retrieves multiple ways, scores relevance, and gates on a quality check before responding. It cost about 3.5x more per query and added 10 to 20 seconds. The point of the Compass is to tell you which workloads are worth that price.
The six axes
For each workload, score it toward A (lean single-shot) or B (lean verify-before-answer).
| # | Axis | A: single-shot is fine | B: you need verify-before-answer |
|---|---|---|---|
| 1 | Complexity | Answer lives in one place | Answer is stitched across multiple sources |
| 2 | Risk | Low cost of an incomplete answer | High cost: regulatory, financial, legal |
| 3 | Latency | Must be instant | Can absorb 10 to 20 extra seconds |
| 4 | Cost | 3.5x per query is fine at this volume | 3.5x is significant at this scale |
| 5 | Corpus coverage | Retrieval is reliable and complete | Coverage is thin or uneven |
| 6 | Auditability | Nobody will need to defend it later | Someone must trace how it was reached |
Notes per axis:
- Complexity. Multi-part questions are where single-shot quietly fails, because it answers from the first hit and stops.
- Risk. The deciding question: what does a confidently incomplete answer cost here? Calibrate the architecture to that, not to the average case.
- Latency. Real-time surfaces often cannot tolerate the extra seconds. Batch and back-office work usually can.
- Cost. At low volume the 3.5x is rounding error. At massive scale it is a real budget line. Decide deliberately.
- Corpus coverage. Counterintuitively, weak coverage argues for verification, because the first retrieved chunk is more likely to be partial.
- Auditability. If an answer may be read back to you by an examiner, the verify pipeline is not overhead. It is the audit trail.
How to read your tally
- Mostly A: answer immediately. Single-shot is the right tool, and adding verification is over-engineering that costs money and latency for no gain.
- Mostly B: use verify-before-answer. The extra cost and seconds are insurance, and the failure they prevent is an examination finding, not a typo.
- Mixed, but B on Risk or Auditability: lean B. Those two axes are veto-level in regulated work. A high score on either outranks a low score on cost or latency.
A worked scoring, end to end
Here is the tool in motion on one workload: an AI feature that answers "what disclosures must we include in this quarter's investor report."
| Axis | Score | Why |
|---|---|---|
| Complexity | B | Disclosures are governed by several overlapping rules |
| Risk | B | An omission is a regulatory and reputational hit |
| Latency | B | A report is not real-time; 20 seconds is fine |
| Cost | A | Low query volume; 3.5x is negligible here |
| Corpus coverage | B | Guidance is spread across documents of varying quality |
| Auditability | B | The output may be examined later |
Tally: five B, one A. The single A is on cost, which is the weakest veto of the six. Verdict: verify-before-answer, without hesitation. Notice how the tool resolves the one dissenting axis. Cost says "single-shot is cheaper," but Risk and Auditability are veto-level in regulated work, and they both say B. The Compass is explicit that a high score on Risk or Auditability outranks a low score on cost or latency, so the answer is not a coin flip. It is determined.
Now flip one input. Imagine the same report feature, but for an internal draft that a human will heavily review before anything ships, with no external audit. Auditability drops to A and Risk softens. Suddenly the tally is closer, and a faster single-shot draft that a human polishes may be the better call. Same feature, different context, different answer, and the Compass shows you exactly which axis moved.
Frequently asked, briefly
- Is verify-before-answer just RAG with extra steps? It is RAG with a completeness gate. The extra steps are reformulation, multi-retrieval, scoring, and a quality check before responding. Those steps are the difference between an answer that ends and an answer that is complete.
- Can I run it only when needed? Yes, and you should. Score the workload, route the high-risk queries through verification and let the rest answer immediately. The Compass is a routing tool, not an all-or-nothing switch.
- Does a bigger model change my scores? No. A bigger model makes a single-shot answer more fluent, which raises the risk that an incomplete answer reads as authoritative. The axes measure the workload, not the model.
- What if I score it wrong? The axes are a forcing function, not an oracle. The value is that they make you ask "what does an incomplete answer cost here" out loud, with a team, before you ship. A rough score you can defend beats a confident gut call you cannot.
- How often should I re-score? Re-score when the stakes of a workload change: a new regulation, a move from internal draft to external filing, a jump in volume that makes the 3.5x cost real. The same feature can sit on opposite sides of the line in two different contexts, so the score belongs to the use, not the endpoint.
One more framing that helps teams adopt this. The Compass is not asking "is our AI good enough." It is asking "what is the cost of being confidently incomplete on this specific workload, and can we defend the answer later." Those are product and risk questions, not model questions, which is exactly why a model upgrade never settles them.
The mistake the Compass prevents
The most expensive AI mistake is not choosing the wrong architecture once. It is having no method, so you default to fastest-and-cheapest everywhere and find the completeness gap during an audit instead of during design. And note what the Compass does not tell you to do: it does not tell you to buy a bigger model. A bigger model is still single-shot. It makes an incomplete answer more fluent and more convincing, not more complete. The fix is architecture, applied where the six axes say it is warranted.
I turned these six axes into a one-page Architecture Selection Compass you can fill in: tick A or B per axis, tally, land on a recommendation. Comment COMPASS and I will send it over. Connect with me so it can reach your inbox.
Which of the six is the deciding factor for your stack right now?
