DSH plugins for Vision & OCR

Vision plugins give a text-only model a way to look. The strongest implementations do not just describe an image, they return structured evidence the model can reason over: extracted text with positions, layout regions, colour values, and crop coordinates it can request next.

Prefer plugins that treat images as ordinary tool calls instead of a special turn type, and check whether the vision backend is local, free, or requires your own key. For screenshot-heavy work, confirm that long images are tiled rather than downscaled into illegibility.

How to evaluate

Three axes matter. Backend: local model, hosted provider, or an anonymous keyless fallback — each has a different privacy and reliability story. Output shape: free-text caption versus structured JSON the agent can chain. Cost and limits: per-image fees, resolution caps, and whether a long screenshot is segmented.

Also check integration style. A bridge that ships images out and brings text back keeps the harness text-native. A plugin that swaps the whole turn to a vision model changes routing and may bypass tools you rely on.

What good looks like

A strong vision plugin is explicit about evidence. OCR returns text with coordinates. UI inspection can name regions and request crops. Pixel-diff or screenshot comparison states its tolerance. Keys stay in configuration; anonymous endpoints are labelled as non-private when that is what they are. Long documents are tiled so the model still sees the bottom of a page.

This directory’s curated vision listings already show the quality bar: structured returns, stated backends, and clear warnings when a path is not private.

Common failure modes

The usual failure is a pretty caption with no positions, so the agent cannot act on “the button is around here.” The second is an anonymous vision fallback treated as safe for sensitive screenshots. The third is downscaling a tall UI capture until the text is mush, then trusting the hallucinated summary of what it said.

Choosing between vision plugins

Do not install three bridges that all claim image understanding. Pick one evidence contract per workflow — coordinates for UI work, long-page OCR for documents, pixel diff for regression — and only add a second when the first fails a concrete task you care about. Compare layouts on the comparison pages before stacking providers.

If your model already accepts images natively, the gap is usually citation and review tooling, not another vision adapter. In that case prefer plugins that turn screenshots into reviewable artifacts rather than another “look at this” turn.