Turning a screenshot into a task means reading the text in the image and then working out what the task is. OCR handles the first part and a language model handles the second. Used alone, each one falls short in a different way.
Reading the text and finding the task
Reading the text is mechanical: turn pixels into characters, in order, line by line. OCR engines have done this well for years, and the clean fonts in a screenshot are easy for them.
Finding the task takes interpretation. Something has to decide which line is the request and which is the signature, what "by Friday" refers to, and who is asking whom. OCR does none of that. It returns the text.
Language models are good at the interpretation. When they read the image themselves, though, they can get small text wrong.
What OCR alone gives you
An OCR-only tool gives you every line it found. You still have to read it, find the request, write a title and set a date. It saves you retyping and nothing else.
The output also includes everything around the message: the channel name, timestamps, the reply count, the "View
thread" link. If nothing decides which line is the request, a tool that makes tasks from OCR output ends up with
titles like #design-reviews 11:32 AM 14 replies Priya.
What goes wrong with a vision model alone
The next idea is to send the image to a vision model and let it do everything. That mostly works, but it makes a few kinds of mistake on screenshots full of text:
- Wrong text that looks right. With a small font or a compressed image, the model can write a plausible word that isn't there. "Send the Q3 deck to Nora" becomes "Send the Q3 deck to Norah", and the task names the wrong person.
- Missing numbers. Dates, amounts, issue keys and version numbers can get dropped. "Follow up on APP-4719" becomes "Follow up on the issue".
- Interface text in the task. A "Send" button or an unread count ends up in the title, because nothing tells the model which text is the message and which is the app around it.
How Capter handles it
Capter doesn't run a separate OCR step. It sends the screenshot to one AI vision model, with rules that make it read carefully before it interprets anything:
- Only write what's visible. A name, date, number or quote can go into the task only if the model can point to those exact characters in the image, or you wrote it in your note or your personal instructions. Otherwise the task stays generic: "the sender", "the article". The rule is there to keep misread names and numbers out of your tasks.
- A busy screen is not a request. Menus, sidebars, unread badges and log lines don't count as tasks. If the screenshot has no request in it, the model is told to return no task instead of making one up.
- Then write the task. From what's left, the model writes a title, a description when there's detail worth keeping, a deadline when the text gives one (worked out in your time zone), and tags based on the names and descriptions of your tags.
- Personalization, while it's on. With "Personalize AI suggestions" on, Capter learns from the edits you make to its drafts. You can also write up to 2,000 characters of personal instructions.
The AI personalization docs cover what is sent to the model, what is stored, and how to turn personalization off. Screenshots are never used to train AI models.
Checking and fixing a task
The model still gets things wrong sometimes. The screenshot is saved with every task, so you can compare the two, and you can edit the title, deadline, description and tags. With personalization on, Give feedback on this draft in a task's More menu lets you tell Capter what it got wrong.
Capter is a to-do app that writes tasks from your screenshots. You can use it for free.