The goal wasn’t “run Llama at home for its own sake.” It was: make an entrepreneur’s existing notes and PDFs answerable, privately, on infrastructure they can maintain.
Design choices that mattered
Separate storage from generation. Notes live in git-backed repos. Vectors live in a dedicated database on the NAS. Chat and workflows run on a small always-on server. That split keeps backups honest and failures diagnosable.
Embeddings local, generation routed. Local embedding models are enough for retrieval quality. Generation can stay on fast external APIs for day-to-day use, with a documented path to more local inference later. The win is ownership of the archive — not purity theater.
Incremental indexing. Full reindexes are fine once. Daily life needs change detection, manifests, and a trigger after capture so new clips show up in search within a minute.
Workflows for real habits. Quick capture from phone and terminal. Web clipper. Scheduled research digests. A morning briefing that reads tasks and projects without reinventing a productivity app.
What we measure
Notes queries dropped from tens of seconds (when everything ran locally on CPU) to under a second with routed generation. Monthly cost for routine use stayed near zero. More important: the system is documented — collections, scripts, failure modes, upgrade path — so it isn’t tribal knowledge.
Related
See the mature project write-up for the public-facing story. This note is the builder’s view: decisions, not a dump of internal IPs and keys.