File Paths, Not Base64: The MCP Contract Behind 23 Screenshot Tools

We almost shipped base64.

Every author of an image-producing MCP server makes one decision in the first hour, usually without noticing: what does a capture tool return? The "obvious" answer is to return the image, encoded as base64 text. Obvious, here, means nobody ran the token math. Heretic Lazy Shot's MCP server ships 23 tools (7 capture, 8 manage, 5 markers, 3 OCR) and every one of them returns a file path instead. That's a hard contract, not a habit.

This sounds like a minor detail. It decides cost, latency, reliability, and which workflows are possible at all.

The token math nobody runs

Start with the naive path. A capture_screen tool grabs a display and returns the PNG as base64 inside the tool result.

A routine full-screen capture weighs 500 KB to 2 MB. Base64 inflates bytes by a third, so the tool result becomes 670 KB to 2.7 MB of text landing in the conversation as one payload. If the client treats it as text, that's on the order of a hundred thousand tokens for a single screenshot. One image can swallow most of a context window by itself. The workflow doesn't degrade. It dies.

The less naive path converts the payload into a proper image block for the model's vision input. Better. A resized full-screen shot still costs roughly 1,500 to 2,000 vision tokens on every model turn where it stays in context. Capture ten windows in a batch job and the bill runs 15,000 to 20,000 tokens for pixels the agent may never look at. Add the cost of pushing megabytes over a protocol built to be lightweight.

Now the file-path version of the same call:

{
 "id": "a1f3…",
 "keyword": "checkout-cart",
 "file_path": "~/.heretic-lazy-shot/screenshots/2026-07-27_checkout-cart.png",
 "width": 2560,
 "height": 1440
}

Call it thirty tokens. The agent knows the capture worked, knows where the file lives, knows its geometry. It can hand the path to a doc generator, an upload step, another tool, or the model's own vision, if and when reading the image is actually required.

Base64 as a default isn't a convenience. It's a tax.

Sending pixels to an agent that asked for a path is delivering the whole lumber yard when the carpenter wanted a hammer.

Reference vs. pixels: what agents actually do with screenshots

The whole design rests on one observation: most agent operations on screenshots don't need to see them.

Walk a real documentation workflow. Capture a window, name it, place numbered markers, insert it into a guide. Four operations. Zero of them require the model to see a pixel. Capture returns metadata. Naming is a library operation. Markers take coordinates or a preset. Insertion takes a path. The image gets referenced, organized, and transformed. Not interpreted.

Interpretation is real, and it's the minority case. Naming a shot doesn't need vision. Asking whether the submit button is visible does. The client reads the file and feeds it to the model's vision exactly once, exactly when asked.

Pixels are a last resort, not a default. Base64-by-default gets the order backwards and charges every operation the full price of the rare one.

The rest of the contract: errors and logs

A file-path contract only works if the agent can trust the paths. That drags two less glamorous decisions into scope.

Errors come structured and honest. Every Lazy Shot MCP tool fails with a machine-readable code, not prose. When the optional Window Activity Tracker is off, the tracker-dependent tools (list_tracked_windows, capture_tracked_window) return a clean feature_disabled error. The tempting alternative is to return an empty list and stay quiet. That reads as success and sends the agent down a false branch.

An empty list isn't a graceful fallback. It's a lie with good manners.

An agent recovers from a named error. It never recovers from a plausible lie.

Every operation leaves a trail. A batch job fails at 2 a.m. We open ~/.heretic-lazy-shot/logs/. One shared rotating file, heretic-lazy-shot_<timestamp>.log, rotated at 10 MB or 24 hours. The MCP lines carry a [MCP] tag. We find the server start, the port conflict, the tool call, the failure. The transcript exists.

That transcript exists because the results are small. Paths log cleanly. Base64 doesn't. When every tool result is a short line of metadata, logging everything costs nothing, and the log stops being the first thing that breaks under load.

Steelmanning base64: when the opposite design is right

The file-path pattern has a real precondition: the tool and the client share a filesystem. Lazy Shot's MCP server runs on localhost next to the desktop app, so a path under ~/.heretic-lazy-shot/screenshots/ means something to every consumer. Break that assumption and the trade-offs move.

  • Remote MCP servers. A path on a machine the client can't reach isn't a reference. It's noise. The answer there is a fetchable reference, a URL or a claim-check ID. Still reference, not payload, with different plumbing. Base64 becomes the last resort when no shared substrate exists at all.
  • Vision-first tools. Projects like Peekaboo bundle capture with visual question answering. The agent asks what's in this window and gets an answer, not an image. A different design solving a different problem. Notice the shared trait: raw image data still doesn't flood the context. The edge distills it into text. The common enemy is unexamined base64 in the middle.
  • One-shot pipelines. If the whole workflow is capture, look, done, with no library and no reuse, the ceremony of paths buys less. Fine. Real workflows accrete.

A checklist for MCP authors shipping image tools

  1. Default to references. File paths on a shared filesystem. URLs or claim-check IDs across machine boundaries. Payloads only when no reference substrate exists.
  2. Make pixels opt-in. If interpretation matters, expose it as an explicit step, never as a side effect of capture. Lazy Shot ships three OCR tools (ocr_screenshot, ocr_image_path, list_ocr_languages) for exactly this: reading on demand, over a reference.
  3. Return rich metadata with the reference. Dimensions, format, timestamps, a human-recallable handle. Short memorable keywords instead of UUIDs, so the agent recalls the checkout shot as checkout-cart. Costs nothing. Changes how usable the library is.
  4. Name your failures. Structured error codes for every branch, including feature_disabled for anything behind a flag. Empty success is a lie.
  5. Log everything, because small results make it cheap. One shared file, rotation by size or time, a tag per subsystem. The future 2 a.m. reader of that log is the customer.
  6. Never mutate inputs. Markers fork, never mutate. Transformations write a copy and return the new reference. Agents make mistakes at machine speed, and originals are the one thing nobody can regenerate.

Where this lands

Heretic Lazy Shot's MCP server is the working proof of the pattern: 23 tools across capture, library management, markers, and OCR. Every one returns paths and metadata. Keyword recall, forking transformations, structured errors, full logging, all of it running locally next to the desktop app humans use on the same library.

The design isn't clever. It's the token math taken seriously.

If you build an image-producing MCP server, steal the checklist. If you'd rather use one, the full tour of Lazy Shot's screenshot MCP server is here. Either way, count the tokens first.

Heretic Lazy Shot is a screenshot, annotation, and beautification tool for macOS and Windows, built for humans and agents. One-time license: €14.99 for one device or €29.99 for three. No subscription, no account, no telemetry. Activation is online, with an offline grace period after. The trial ships inside the download, no card. Try Lazy Shot →

Filed under:Heretic Lazy ShotEngineering
Tagged:

Related posts