Orion
← Back to work

LLM Agents · Research tooling · 2026

Paper Distiller — A Conversational arXiv Research Agent

Role
Design · Implementation
Stack
Python · LLM · RAG · SQLite / FTS5
Context
Open source · PyPI / MIT
Year
2026
  • ~1.7M arXiv
    Local mirror
  • 7
    LLM tools
  • ~¥0.04 / paper
    Deep distill cost
  • 436
    Tests

A conversational agent: search → deep-distill → cross-reference proofs, turning arXiv papers into a searchable, interlinked markdown knowledge base (Obsidian-compatible) backed by a ~1.7M-paper local mirror.

Background

The slow part of reading papers is truly absorbing one and connecting it to what you have already read. I wanted a conversational assistant: tell it what you want in natural language, let it choose which tool to call, and have it distill papers into a searchable, interlinked knowledge base.

Approach

Make "read — distill — link — query" into seven LLM-callable tools, sit on a ~1.7M-paper local arXiv mirror with SQLite + FTS5 for millisecond search, and emit a 12-section deep distillation plus a theorem / technique proof sidecar for every paper.

  • Search — keyword + semantic hybrid retrieval over the ~1.7M-paper local mirror
  • Deep distill — 12 sections, fixed structure, so cross-paper reuse becomes possible
  • Proof sidecar — theorems / techniques pulled out; later distillations auto-fetch related prior ones
  • QA — precise question-answering on already-distilled papers
  • Long research — batch synthesis across papers and topics

Key design

The vault's notation and naming converge over time — the compounding effect this tool is most proud of.

  • Auto-pull-prior on each distillation — feed related existing theorems / techniques to the model, naming aligns naturally
  • Obsidian-compatible output — distill cards are markdown + back-links, drop straight into a vault
  • Conversational entry point — no CLI flags to memorise; tell the agent what you want
  • Cost-controlled — ~¥0.04 per deep distill, which is what makes running the whole library meaningful

Lessons

Research tooling falls easily into the "many features, no one uses any of them" trap. The trade-offs here.

  • Entry funneled to dialogue — let the LLM pick the tool, not the user memorising flags
  • Distill format locked to 12 sections — looks like a limit, actually what makes cross-paper reuse work
  • Local mirror is non-negotiable — hitting arXiv API in real time is slow and unstable

Status

Published on PyPI (MIT) with 436 tests; supports multiple LLM providers; cost-controlled — a deep distillation runs about ¥0.04 per paper.