Why I Built a 5-Layer AI Memory Stack on an M1 Mac (And Why Stateless Chatbots Are Dead)

Every AI agent you've ever talked to has a problem: it forgets. You spend 20 minutes building context, the session ends, and the next conversation starts from zero. That's not intelligence — it's a calculator with a chat interface.

I refused to accept that limitation. So I built a 5-layer memory infrastructure that gives my AI agents persistent, searchable, cross-session memory — running entirely on an M1 Mac with zero cloud cost.

Most AI agents today are stateless. They get a prompt, they respond, they forget. This works for one-off questions. It fails completely for business operations that span days and weeks, financial tracking that requires historical context, research projects that build on previous findings, and personal assistants that need to know who you are.

The fix isn't a bigger context window. It's a memory architecture.

Layer 1: Honcho — Vector Embeddings (pgvector)

Every conversation, every document, every decision gets converted into a vector embedding using nomic-embed and stored in pgvector running inside Docker. This means my agents can search by meaning, not just keywords. Ask "what did we decide about the trading stop-loss?" and it finds the relevant context even if those exact words were never used.

Tech: pgvector, nomic-embed, Docker, PostgreSQL

Layer 2: Session Cache (Redis)

Active session state lives in Redis. When an agent is mid-task, it doesn't need to re-query the vector database — the current context is cached and instantly accessible. This is the difference between an agent that responds in 200ms and one that takes 8 seconds per tool call.

Tech: Redis, Docker

Layer 3: Full-Text Search (SQLite FTS5)

Across all my agents, there's 359 MB of raw conversational history. SQLite FTS5 indexes every word, making it searchable in milliseconds. When I need to find a specific decision from 3 months ago, I don't scroll — I search.

Tech: SQLite FTS5

Layer 4: Knowledge Catalog (Obsidian Vault)

This is the human-in-the-loop layer. A 102-page Obsidian vault serves as the structured knowledge catalog — project statuses, daily logs, mistake records, session summaries. Agents write to it. I read from it. It's the shared memory between human and AI.

Tech: Obsidian API, Markdown

Layer 5: Built-in Memory

The smallest but most critical layer. A 2,200 character hard limit stores only the essential routing information — Telegram IDs, email preferences, critical file paths. Injected into every single prompt automatically.

Tech: Hermes built-in memory system

Why This Matters

Memory is what separates a chatbot from an agent. Without memory, you're just having the same conversation over and over. With a proper memory stack, your agents build on previous work, learn from mistakes, and maintain context across days, weeks, and months of operation.

And it all runs on a single M1 Mac. No cloud. No monthly bills. No vendor lock-in.