Choosing between the tools

Which tool to use is the question everybody asks first and it is close to the least important. The gap between the current leaders is smaller than the gap between using one thoughtfully and badly, and whatever ranks top today will not in six months.

Why this article names nothing

Deliberately. Model names, context limits, prices and which one is ahead all change on a scale of months, and an article that lists them starts lying to readers almost immediately. What does not change is how you decide, so that is what is here. Treat any comparison you read, including the ones with tables, as a snapshot of a week.

The decisions that actually matter

1. 01Where does your data go. Whether inputs are used to train the provider's models, how long they are kept, and which country they sit in. This is a question about your customers' data as much as your own, and the answer belongs in your privacy policy.
2. 02Does it work in your languages. Quality drops away from English, and it drops unevenly. Test with your own real text, not with a demo.
3. 03Does it go and look things up, and does it tell you where. A tool that cites a page you can open is a different proposition from one that recalls. This single property changes how much checking you have to do.
4. 04Can you get your work out. Prompts, documents and history. Anything you cannot export is something you will rebuild when you move.
5. 05What does it cost at the volume you will actually reach, not at the volume in the example.

How to actually compare two of them

Not by reading comparisons. By building a tiny test set and keeping it.

1. 01Write down ten real tasks from your own work, with the inputs you really have and the output you would accept.
2. 02Run all ten through each candidate, unchanged, on the same day.
3. 03Score them on whether you would have sent the result, not on whether it was impressive.
4. 04Keep the set. When something new appears, you have a half-hour answer instead of an argument, and you can tell improvement from marketing.

That test set is the most valuable thing in this article. It costs an afternoon once, it answers every future version of the question, and almost nobody builds one.

Switching costs, honestly

Lower than people fear for conversation, higher than people expect for anything built. A prompt tuned against one model is not portable advice, and an automation that depends on a particular output shape will need rebuilding. So keep conversation tools loose and change them freely, and when you build something that runs by itself, assume you will rebuild it within two years and decide whether it still pays at that rate.

One tool or several

A question worth settling early, because the answer is not the obvious one. Running two is often better than running the best one, for reasons that have nothing to do with quality.

• Disagreement is information. On a judgement where you cannot easily check the answer, two tools that say the same thing is weak evidence and two that differ is a signal to go and look properly. One tool gives you no such signal.
• They fail at different times. An outage in the middle of a working day matters more to a small business than a few points of benchmark difference.
• It keeps you honest about lock-in. Anything that works in only one of them is a dependency you have taken on without deciding to.

Against that, two subscriptions cost twice as much and split your attention, so this is a judgement about whether the work is load-bearing. If a day without the tool is an inconvenience, pick one. If a day without it stops something that earns money, having a second account you barely use is cheap insurance.