Putting a model inside a workflow
The useful question is which of its failures you can afford. Put it where a wrong answer is visible, and design the position rather than the prompt.
Language models are now a component you can put inside a workflow, which is genuinely new and widely misapplied. The useful question is not what they can do but which of their failures you can afford.
The jobs they do reliably
Broadly, tasks where the input contains the answer and the job is to reshape it.
- Classifying: which of these five categories is this message, is this urgent, is this a complaint.
- Extracting: pull the date, the amount, the address out of this unstructured text.
- Summarising something you will also read, where a wrong emphasis costs little.
- Reformatting and tidying: notes into prose, a transcript into bullet points, inconsistent data into a consistent shape.
- Drafting a first version of something a person will then edit.
- Translating, with the same caveat as any machine translation: fine for comprehension, checked before publication.
The jobs they do unreliably
- Anything requiring a fact not in the input. A model asked for a statistic will produce a plausible one, and plausible is the problem.
- Arithmetic, over anything more than a few numbers.
- Deciding, where the decision has consequences and nobody checks it.
- Anything regulated: medical, legal, financial, tax. The answer will be confident and the confidence carries no information.
- Consistency across many runs. The same input can produce differently shaped output, which breaks whatever is downstream.
Making the output usable by a workflow
- Ask for a fixed shape and validate it when it arrives, rather than assuming it came back as requested.
- Constrain the answer where you can: one of these five words, not a free description.
- Give it a way to decline. Unclear is a correct and useful answer, and without that option a model will guess.
- Handle the invalid response as a normal case, routing it to a person rather than failing.
- Log the input and the output, so when something odd reaches a customer you can see what produced it.
The two obligations
First, what you put in. Customer data sent to a third-party service is a processing decision with privacy implications in most jurisdictions, and it needs to be covered by your own privacy notice and whatever terms you agreed. Read what the provider does with it, including whether it is retained or used for training, before routing anything personal through it.
Second, what comes out. Published material carries your name regardless of what wrote it, and unreviewed generated text is where invented facts enter a business. Search engines have been explicit that the problem is not the involvement of a model but publishing at scale without adding value, and the same judgement serves customers.
Cost, speed and limits
Practical constraints that catch people after the demonstration worked. A model step costs money per run and takes seconds rather than milliseconds, so putting one inside a workflow that fires on every message has a bill and a delay attached.
- Price it at your actual volume before building, not at the volume you tested with.
- Filter before the model step, so it only runs on the cases that need it.
- Expect the occasional failure or refusal from the service, and handle it as a normal path rather than as an error.
- Do not put one in front of a customer waiting for a page to respond unless you have measured how slow it is at a busy moment.
What to tell people
A question with both a courtesy and a compliance side. If a customer is talking to something automated, say so, plainly, and give a route to a person. Pretending otherwise is the thing people resent most, and disclosure obligations around automated interaction are being written into law in a growing number of jurisdictions.
Where personal data is involved there may also be a duty to say so in your privacy notice, including who the data is shared with. The safe position for a small business is to name it: this is handled automatically, here is how to reach somebody, here is what we do with what you send.
Checking it is still right
A model step can get worse without anything in your workflow changing, because the service behind it is updated on its own schedule. So keep a small set of inputs whose correct output you know, and run them through occasionally. Twenty examples is enough to notice a drift, and it is the only way you will find out that the classification which was reliable in March has been quietly wrong since July.
Work to do
- For each model step, write what happens when it is wrong. If the answer is nobody notices, add a person.
- Make every model step return a fixed shape, and validate it.
- Check what your provider does with the data you send, and whether your privacy notice covers it.
Sources