The AI runs here.
Including its limits.
Every model PDFMacro uses runs on your machine — either in this browser tab, or on an AI server you run yourself. This page names them, says what each is for, what it costs to download, and where a model small enough to sit beside your document stops being good enough.
Two places, and one that is never used.
The built-in models. Weights are fetched once and cached on the device; after that the document and the model are in the same tab and inference makes no network request at all.
Optional. If you already run Ollama or LM Studio, PDFMacro can use your models instead. Document excerpts then go to that endpoint — your machine — and the code refuses to send them anywhere else.
Never. There is no hosted inference provider in this codebase to send text to — no key, no endpoint, no client. The absence is the mechanism, not a setting.
Three models, named by what they do.
The names below are the ones the app itself uses in its readiness panel. “MiniLM” and “NER” are how two downloads are told apart in code; they are no help at all to someone deciding whether to spend ~110 MB of their connection, so the repository id is kept for auditing and the capability leads.
Understanding what you type in the command bar, and Pre-discovery search
Finding names and other sensitive details in a document, and Privilege review
Draft with AI — turning your instructions into template text, on this device
The command bar assistant's conversational replies, when it cannot match what you typed to a tool
The two default models come to ~155 MB together, and consent is asked per model rather than once for all of them — agreeing to the smaller one for the command bar is not agreeing to the larger one for entity recognition. They are different sizes bought for different reasons.
Readiness also means “everything you can actually use is ready”: both features the second model powers are part of the licence, so a free account is never told to fetch it.
Weights live in your browser's Cache Storage, on this device — not in your account. So they stay put, and they also stay subject to your browser: clearing site data, a private window, or the browser reclaiming space will remove them, and you will be asked once more rather than charged the bandwidth silently.
Your laptop decides which model you get.
The drafting model is not one model. Before anything downloads, the app probes what this machine is — whether a WebGPU adapter exists and how many logical cores there are — and picks from a ladder. A WebGPU machine runs the model through WebLLM; a machine without one runs a smaller model through WebAssembly, which is the same runtime the two default models already use.
Then a memory gate applies. Drafting runs beside document work — page render buffers, scan workers and undo history are all still resident — so a model may claim only a fraction of reported device memory, and must leave browser storage headroom after it downloads. When the preferred model does not fit, the ladder walks down and says so. When nothing fits, the feature declines with a reason you can read, rather than failing halfway through a draft.
Resolved by the same function the app calls before asking your consent, so the size you are quoted is one this browser can actually store.
A model that fits beside your document is not a model that fills a data centre.
They are smaller, and it shows.
The models here are measured in hundreds of megabytes to a couple of gigabytes. Hosted frontier models are orders of magnitude larger and run on hardware you do not have. On long or subtle documents you will find the difference: shallower reasoning, weaker recall across many pages, and prose that needs editing.
What you get depends on your machine.
The same feature is a different experience on a WebGPU workstation and a four-core laptop — a different model, more slowly. That is not a licence tier; it is physics. The tier is probed rather than asked about, and the ladder above shows what each shape is offered.
We have watched one fail.
A previous drafting model was removed from the ladder after three consecutive real-device drafts degraded into fluent nonsense — Arabic characters dropped into an English demand letter, invented word endings. That is what a small quantized model failing looks like, and it is why the current ladder prefers the numerically safer build even when a cheaper one would fit.
Reading, not writing. Matching what you typed against what a document says. Finding names, addresses and other sensitive details across hundreds of pages without any of it leaving the tab. Those are bounded jobs with checkable output, and a small model does them well — which is exactly why they are the two that download by default and generation is a separate, opt-in choice.
If you want generation at a quality small models cannot reach, the answer is not a cloud key. It is the next section.
Point it at a bigger model you already run.
Deliberately advanced and deliberately subordinate: it is collapsed by default in the AI panel and it never competes with the one-click built-in setup. It is for someone who already runs a local model server and knows why.
Paste an endpoint — http://localhost:11434 for Ollama, http://localhost:1234 for LM Studio — and press Test connection. That click is the only thing that fetches. Nothing dials out on page load, on import or on a timer, and a saved endpoint is remembered as text that becomes a live connection only when you press the button again.
Once connected, Chat with PDF and Summarize send document excerpts to that endpoint and nowhere else. That is enforced rather than intended: every request URL is checked against the origin you configured, and one outside it throws instead of being sent.
PDFMacro never pulls models onto your server. It lists what is already installed; when a server has none, it prints the command for you to run yourself, sized for the machine it detected:
ollama pull qwen2.5:7b4.7 GB · your machine can run a 7B model comfortablyollama pull llama3.2:3b2.0 GB · a good balance for this machineollama pull llama3.2:1b1.3 GB · small enough to run smoothly hereconnectedThe server answered and listed its models — exactly what `ollama list` shows, with no editorialising. Press “Use this server” and inference moves there for Chat with PDF and Summarize.
origin-rejectedSomething is listening and it refused this page's origin. For Ollama, set OLLAMA_ORIGINS to this app's origin and restart it. In LM Studio, enable CORS in the server settings.
unreachableThe server is not running, the port is wrong, or the browser blocked the request before it left — a page served from a public address can be stopped by the browser's private-network rules before the server's own configuration is even consulted.
The middle case is the one worth knowing about in advance. A server that is running but has not been told to accept this app looks, to page JavaScript, exactly like a server that is not running — both surface as the same fetch failure. A second probe separates them, because a present-but-refusing server still returns an opaque response while an absent one throws. That is what lets the app name the actual problem instead of shrugging.
Questions about the local AI.
Which AI models does PDFMacro download?
Do the built-in models write text, or only read it?
What does “local” actually mean here?
Are these models as good as ChatGPT or Claude?
Why does my own server need OLLAMA_ORIGINS set?
Will PDFMacro download models to my Ollama server?
Watch it download, then pull the plug.
The models come down once, in the open, with the size on the button. After that, disconnect and try the same feature again.