Parth Parekh
All work

Air-Gapped Document Assistant

Local RAG, OCR and a knowledge graph

Year
2026
Kind
Systems
Status
In use
Built at
QED Analyticals

Fully air-gapped in production

An assistant for engineering documents and drawings that runs entirely on local models: OCR, retrieval over a vector store and a knowledge graph, and an agent that cites its sources.

Built for engineering teams that can't send a page to the cloud. Local OCR and vision models read specifications, scans and drawings; text is chunked into ChromaDB and a language model extracts entities and relations into Neo4j. Questions are answered by an agent with native tool calling, which retrieves diverse passages and related entities and streams its answer with clickable citations. In production it runs fully air-gapped; I developed it against a cloud GPU through my Ollama-compatible vLLM proxy.

The problem

Some engineering teams can't send a single page to a cloud model: their specifications and drawings have to stay on machines with no internet connection. They still need to search hundreds of pages, check documents against rules and pull structured data out of drawings. This assistant does all of it with local models: OCR, retrieval over a vector store and a knowledge graph, and an agent that cites its sources.

How it works

  1. 01

    Read every page, drawings included

    Each PDF page is rendered at 2.5x and read by a local OCR model, and if that returns nothing a vision model tries instead. Drawings are rendered at 4x and the model first labels each page as text or drawing: text pages go to OCR, drawing pages get a structured vision description. Tables are split from the prose and parsed into cells, so they come back out as real Word and Excel tables.

  2. 02

    Chunk and embed, keeping the source

    Text is cut into 1,000-character chunks with 200 characters of overlap, so a sentence split at a boundary still appears whole in one of them. A local embedding model turns each chunk into a vector in ChromaDB, and every chunk keeps its file and page, which is what makes a citation possible later.

  3. 03

    Extract a knowledge graph

    A language model reads each slice of a document and returns entities and relations as JSON: systems, equipment, standards, parameters and lessons, connected by relations such as component-of, applies-to and complies-with. They go into Neo4j, with every entity linked back to the document it came from.

  4. 04

    Retrieve two ways

    Vector search uses maximal marginal relevance, which picks chunks that match the question but don't repeat each other, so the context covers more ground. In parallel, entities named in the question pull their direct neighbours out of the graph. Both go into the prompt: the chunks for exact wording, the graph for how things relate.

  5. 05

    Let the model choose its tools

    The chat model works as an agent with native function calling. It can search the knowledge base, read a document, OCR a page, analyse an image, read a spreadsheet or record a lesson learned, choosing tools step by step at low temperature until it has an answer.

  6. 06

    Stream the answer, with citations

    Answers stream over Server-Sent Events, with separate events for status, tool calls, tokens and sources. The model cites with inline markers, and clicking one opens the cited passage, found in the original PDF by matching its text.

Decisions

  • Local models only in production

    The deployed system is fully air-gapped, with every model served inside the network through an Ollama-style API. I didn't have hardware that could run these models while building it, so during development I served them from a cloud GPU through my Ollama-compatible vLLM proxy, and the app couldn't tell the difference.

  • Read the rendered page, not just the text layer

    Scanned specifications and exported drawings often have no usable text layer, so indexing reads what the page actually looks like. A fast text-layer path is still there for clean, born-digital files.

  • Generated output stays editable

    Reviews, extracted tables and bills of materials are stored as structured JSON and shown read-only by default. An edit is validated, regenerates the Word and Excel files with the same builders that made them, and is written to an audit log.