How it works

Reading documents

Every document you add is read on your Mac before anything is filed. This page follows a file from the moment it arrives to the moment it is searchable, and says where the reading can go wrong. It is for people who want to know what the app does with their files, and for developers.

The pipeline

The page works through documents one at a time in the background. For each one:

Store. The original is encrypted and saved as a blob. Originals stay where they were; Ovillo keeps encrypted copies.
Extract text. Digital text is read directly; scans and photos go through OCR. See below.
Tables. If the Apple silicon helper is ready, PDFs and images also go to it for table structure, which the lab reader uses as a second opinion.
Fields. The document type, date, facility and clinicians' names are picked out of the text.
Whose is it? The patient check compares the name on the page with the person whose record this is.
Lab values. The lab reader finds results. Routine ones are filed, marked and undoable; anything doubtful is held. See How the record is built.
Index. The text is cut into passages and added to an encrypted search index.

If you close the window part-way, the next unlock picks up where it stopped. Reading and indexing never need a network connection. Plain text, pages, passages and vectors exist only in memory and in encrypted blobs.

File types

Kind	Files	How it is read
PDF	`.pdf`	pdf.js reads the text layer. A page with no real text is rendered and sent to OCR. Typed values in fillable forms are included. Password-protected PDFs open when you type the password in the app; it is checked on this computer.
Photos and scans	`.jpg` `.jpeg` `.png` `.webp` `.gif` `.bmp` `.tif` `.tiff` `.heic` `.heif`	OCR. HEIC and TIFF are decoded on this computer first.
Word and similar	`.docx` `.odt` `.rtf`	Ovillo unpacks and reads them with its own code. Nothing is executed.
Email and web	`.eml` `.html` `.htm`	Text only. Attached text files in an email are read in place; an attached PDF is listed, and you add it as its own document.
Plain text	`.txt` `.text` `.log` `.md` `.markdown` `.csv` `.tsv` `.json` `.xml`	Read as text. Older text encodings (Spanish Latin-1 files) are handled.

Not read: old .doc, spreadsheets, slides, Pages, Numbers, Keynote, archives, audio and video. A folder drop lists these as left out, with a reason. An older or password-protected Word file gets a message asking you to save it as .docx or PDF without a password. A file named .pdf that is really a JPEG or PNG is read as the image it is. A file is limited to 100 MB, and a PDF to its first 500 pages.

PDF text

For each page, Ovillo asks pdf.js for the text layer. If the layer has fewer than 25 characters, or less than half of them are letters or digits, the page counts as a scan. Judging by the text layer alone means a fillable cover sheet laid over a scanned page does not hide the scan.

OCR for scans and photos

OCR is Tesseract, running inside the page, with English and Spanish together (eng+spa). Other languages are not supported. Scanned PDF pages are drawn at twice size; phone photos are shrunk so the longest side is 2,500 pixels, since more than that does not help.

A phone photo is often sideways, upside down or at an angle, and Tesseract reads those badly. So each page gets a confidence score, and a page scoring under 60 gets a second look:

Apple's reader first, if available. If the helper is ready, it reads the page. Its text replaces Tesseract's only when it has clearly more real words.
Probe the orientations. On a small copy (700 pixels), Ovillo tries the page turned 90 and 270 degrees, upside down, mirrored, and skewed 15 degrees either way, stopping early once a result is good enough.
Re-read the winner. If one orientation is clearly better, that version is read again at full size, and kept only if it is surer and has not lost most of the words.

The extra effort is capped at 20 seconds per page and 90 seconds per document, so a bad 500-page scan cannot run for hours. Pages that are read cleanly cost nothing extra.

Important. OCR can misread. A value from a page that OCR read poorly (under 60 percent, or rescued by Apple's reader) is held for you to check unless a recognised table on the page confirms it. Always look at the page itself, which every record entry links to.

The lab reader

Lab results are read from the document's own text by plain rules, with no model. src/labs.mjs knows 50 built-in tests, with the names English and Spanish reports print for them (for example "ferritin" and "ferritina"). For each match it keeps:

the value exactly as printed, as text, with a parsed number only for comparisons. "0,42" stays "0,42".
the unit as printed, when it is one the app knows.
the reference range as printed, from "70-110", "< 5", "ref: 12 to 300" or similar.
whether the report or the range marks the value high or low: letters such as H, L, High, Low, Alto, Bajo, an arrow, or a value outside its range.

It understands several layouts: a line of text, a result with the unit after the range, and rows of cells. It will not read a layout it does not know; that becomes a new rule, not a guess. Short abbreviations such as K or Na only count when a number follows right after, so a bacterium name or a surname is never read as a result.

Several things make the reader hold a value instead of filing it: a test run on another specimen (urine, sweat, spinal fluid and others, named anywhere on the page or in a table title), a line that prints the value about someone else (a parent, a donor, cord blood), an unknown unit, a value far outside its range, and two readers disagreeing. When the helper supplies tables, the table reader and the line reader are compared; if they read a value differently, both readings are shown. Ovillo also never converts a value in place: another unit is shown beside it.

The patient check

src/identity.mjs reads the patient field of each document, so a relative's report in the same folder does not land in this person's record. It looks for labels such as "Patient", "Patient name", "Paciente" and "Nombre del paciente" in English and Spanish. A bare "Name:" is ignored because letters use it for doctors. The result is one of four states:

match: the name agrees with the person's name, or a name you confirmed as theirs (a maiden name, a hospital's spelling).
none: the document names no patient. It is not held for this reason.
unset: no name has been set yet, so documents wait until you confirm whose record it is.
other: a different name, or a labelled birth date that does not fit. The document is held. Names match when every part of the shorter name appears in the longer one and at least one full part agrees, so siblings who share surnames are told apart by their first names.

A first name that looks like a scan damaging the real one ("Luci" for "Lucía", surnames exact) is held too, but in a gentler "hard to read" state. Only labelled numeric birth dates are read. See Review for how held documents are decided.

Duplicates

Identical files. The file's SHA-256 is compared. A second copy with the same bytes is marked as the same as the first and is not read again.
Probably the same report. Each document keeps a short signature of its lab values. Two documents with the same date, the same place and mostly the same values are noted as probably the same report. This is only a note on the Documents row. Nothing is deleted, merged or skipped.
Similar photos. With the helper, each photo gets a visual fingerprint, and up to three near photos are noted. It is a hint only: a different page of the same template can look closer than a resized copy of the same page.

The local search index

Text is cut into passages of up to about 1,400 characters, with some overlap. Each passage is indexed twice: by words (BM25) and by meaning, using multilingual-e5-small, a small model that runs in the page and covers Spanish and English. The two result lists are merged. The model loads from the app's own files and cannot download anything. The index is stored as encrypted blobs. If the saved index was made by a different model, or cannot be loaded, Ovillo rebuilds it from the stored text. If the model cannot load, documents are still searchable by words.

The local name model

A second small model, bert-base-multilingual-cased-ner-hrl, finds the names of people written without a title ("came with her daughter Lucía Fernández"). Its only job is to teach the scrubber which names to replace before an assistant request leaves. It runs on this Mac and cannot download anything. See Privacy and security.

What the Apple silicon helper adds

The bundled helper, offload, uses Apple's on-device models and exists only in the Apple silicon app, on macOS 27. On an Intel Mac, or an Apple silicon Mac on an earlier macOS, the built-in readers described above do all the work and the helper features are absent. It adds:

A second opinion on poor scans, as described under OCR. Apple's reader is not used for every page, only to rescue poor ones.
Table structure in PDFs and images, for the lab reader's cross-check.
A second name finder, whose names are added to the scrubber's list.
Photo similarity hints, as above.
Dictation and translation: voice notes are turned into text, and document text can be translated, on this Mac.
A one-line description of each document, off by default, shown only in the Documents list, made only while no assistant is connected. It needs Apple Intelligence, can be wrong, and nothing else reads it.

The helper starts on first use, never at launch. It runs inside the macOS App Sandbox with no network permission, so macOS itself refuses any connection it tries. It receives document bytes in memory, through a pipe, writes no temporary files, keeps nothing between requests, and stops when the vault locks or after a minute without a call. The first read after an install can take a long while, because Apple compiles its models then.

Something unclear or out of date? Tell us on GitHub.