Back
EssayDec ’222 min read

Plausible is cheap. Checking isn’t.

ChatGPT’s first week brought delight, viral screenshots and a temporary ban from Stack Overflow. A plausible answer costs almost nothing to make and a lot to check, and the checking lands on whoever reads it.

Typeface

OpenAI put ChatGPT online as a free research preview on November 30. TechCrunch described it as a fine-tuned GPT-3.5, trained with people ranking its responses, and noted OpenAI’s own warning that it can make things up and knows little about the world after 2021. A week later, it’s most of what people in tech are talking about.

On Monday, Stack Overflow temporarily banned answers written by ChatGPT. Its moderators wrote that “the average rate of getting correct answers from ChatGPT is too low.” The answers looked right, and there were a lot of them, and the volunteers who check answers couldn’t keep up.

The same day, Ben Thompson wrote about homework and argued that the real skill now is verifying the answers.

Stack Overflow is the first clear example of a cost that’s going to show up everywhere. A plausible answer costs almost nothing to produce and a lot to check, and the checking lands on whoever reads it.

For most of the internet’s history, a confident, well-formatted answer was a weak but real signal that someone had put in effort. ChatGPT removes that signal. The style of competence is now free, and the substance still has to be checked by someone who knows the subject.

When I worked on Uber Studio, one of its tools was Stub. It let an engineer replace any service’s response with one they wrote, so they could watch what the product did when a dependency sent back something wrong. Most of the bugs worth finding were in what happened next.

Every product built on a model like this one needs the same habit. Assume the model will sometimes return a fluent, confident, wrong answer, and design what happens next:

  • Show where an answer came from whenever there’s a source to show.
  • Make it easy for the reader to say “this is wrong,” and make that report go somewhere.
  • Keep the person who can check the answer close to the decisions that matter, rather than handing them a summary after the fact.

It’s been a remarkable week. I’ve spent hours with ChatGPT, and some of what it does is already useful. But the tools for producing answers just got far better, and the tools for checking them haven’t changed. That gap is where product designers have work to do.