Keep a person on the other end
GPT-4’s first image partner was Be My Eyes, which describes the world for blind and low-vision people. When the user can’t check the answer, the design is the way back to a person.
OpenAI released GPT-4 on March 14, and TechCrunch reported that it can take images as well as text. The first partner for image input was Be My Eyes, an app that connects blind and low-vision people with sighted volunteers over live video.
The same day, Be My Eyes announced a Virtual Volunteer built on GPT-4: point your phone at something and the model describes it and answers questions about it. It’s in a closed beta, free for blind and low-vision users, and it sits inside an app with 6.3 million human volunteers.
This is the first AI launch this year that made me think about the design before the demo.
Who can check the answer
Most of the talk about chatbots getting things wrong assumes the user can tell. Read the answer, compare it with what you know, catch the mistake. That’s a weak safeguard even for sighted people in a hurry. For someone asking what’s in a photo they can’t see, it isn’t a safeguard at all.
When the person using a model can’t check its answer, the most important part of the design is the way back to a human. Be My Eyes has that built in. The model is a new first step in an app whose whole reason for existing is a person on the other end, and its announcement says that when the model can’t help, the app offers to connect you to one.
Detect, then hand off
The pattern is older than AI. On Uber’s experimentation platform, nobody can watch a thousand experiments, so every flag has guardrails: metrics with limits. When one is breached, the system marks the flag and pages its owner, and that’s all it does. The machine notices and the person decides.
For AI tools I’d design the handoff the same way:
- The model says how sure it is in words a person can act on.
- Asking for a person is one tap from every answer, and it never feels like failure.
- The person who takes over sees what the model said, so the user doesn’t have to repeat themselves.
A lot of products will be tempted to build the model and drop the person, because the person is the expensive part. For anyone who can’t check the answer, the person is what makes the model safe to use.