← All writing

Before choosing a model, understand the workflow.

A practical starting point for deciding whether an AI project is worth building.

Start with a decision somebody makes

An AI project becomes easier to reason about when its starting point is a specific piece of work. “Improve productivity” is too broad to test. “Help a support specialist locate the current policy for an unusual request” has a user, an input, and a decision that needs support.

Map what happens today. Who asks the question? Where does the evidence live? Who checks the result? What happens when the information is missing? The answers define the system more usefully than a model comparison does.

Write down the cost of being wrong

A suggested document and an automatically approved transaction carry different consequences. Put the failure cost beside the desired outcome. This helps decide whether the system should retrieve information, draft a response, ask a clarifying question, or leave a decision with a person.

For a first pilot, choose a boundary that people can inspect. A narrowly scoped assistant with visible sources gives the team something concrete to evaluate. Expanding its authority should be a separate decision supported by evidence.

Build a baseline before a prototype

Collect representative examples of the current workflow, including difficult and ambiguous cases. Record how the task is completed today and what counts as an acceptable result. A manual process, keyword search, or a simple rules-based approach can be a useful baseline.

Then write a small experiment: which change are we testing, against which examples, and who will judge the result? Agree on a stopping rule as well as a success condition. If the data is inaccessible or the workflow does not justify the added complexity, the experiment has still produced a useful answer.

The first deliverable is clarity

A short project brief should identify the user, workflow, available data, expected benefit, review responsibility, and test set. It should also name what the system will not be allowed to do.

This brief is not a substitute for implementation. It makes implementation easier to evaluate. The next technical decision can then be explained in terms of the work it improves.

KEEP EXPLORINGAn answer is only as useful as its evidence.Design your evaluation around what can go wrong.