Outcome. You can design an experiment comparing full image, cropped image, extracted text, and structured UI data for a document or screen task.
A PDF or screenshot mixes several problems: text recognition, reading order, layout, tables, charts, icons, and spatial relationships. APIs may extract an embedded text layer, render pages to images, split images into tiles or patches, or combine these methods. Raw pixel resolution therefore does not translate into a universal token or accuracy formula.
Failure often begins before reasoning. Tiny text may be downsampled; a wide screenshot may make a relevant button occupy very few pixels; OCR may scramble columns; a scanned PDF may have no text layer; a chart legend may be visually distant from its data. Cropping, higher detail, page selection, OCR, DOM/accessibility data, or structured table extraction can change the result more than switching to a larger model.
Use evidence-preserving outputs. Ask for quoted text, page number, bounding box, or identified UI label, and verify them. For computer interfaces, prefer stable semantic selectors or accessibility data when available; pixel coordinates are fragile.
Mental model. First test whether the system perceived the evidence correctly; only then judge its reasoning about that evidence.
Evidence trail — reviewed 23 July 2026. GPT-4V’s system card documents visual limitations: https://openai.com/index/gpt-4v-system-card/. Google’s live document-processing guide is a current implementation example: https://ai.google.dev/gemini-api/docs/document-processing.
Test four inputs: full PDF, rendered page, crop around the totals, and extracted text/table. Ask for total, currency, page, and exact evidence. Repeat on five invoices containing discounts, tax, and similar subtotal/total labels. Record accuracy, latency, and input cost.