Azaz Ahmed Lipu

I Built a Shared Memory System for AI. Here's Why.

There is high probability that a small model with good memory will beat a massive model without it.

Every AI today has amnesia.

AI is shifting fast. Agents are becoming the core interface. They write code, manage projects, answer questions, make decisions. But there is a fundamental problem nobody has properly solved yet.

Every AI agent today forgets.

Claude remembers things inside its own session. ChatGPT has its own memory. Cursor knows your codebase but nothing about your preferences from yesterday's ChatGPT conversation. Three different agents. Three different memory silos. None of them share anything.

You repeat yourself constantly. They contradict each other. There is no single source of truth about you, your project, or how you work.

Here is the thing. A 7 billion parameter model with proper context management will outperform a 70 billion parameter model without it. Context is the multiplier. Memory is the moat. Without it, even the smartest agent is just guessing.

I have been researching this problem and built a prototype. It is called OpenMemory.

What It Does

OpenMemory is a shared memory system that sits between any AI agent and a persistent database. Think of it like DNS for AI memory. The agent does not need to know how it works. It just asks "what do you know about this user?" and gets back a ready to use context string.

Three parts make it work.

First, the memory store. It holds memories the way humans think about information. Each memory has a type (fact, preference, belief, task), a confidence score, an importance weight, and a status that evolves over time. Active becomes stale becomes superseded becomes archived. Nothing is ever deleted. Every change is logged forever in an immutable event log.

Second, the retrieval engine. When an agent asks a question, the engine uses four signals at once to find the most relevant memories. Semantic similarity (what does this text mean). Recency (how new is this). Importance (how much does this matter). Access frequency (how often is this used). It is not just vector search. It is a composite score that feels more intelligent than mechanical.

Third, the standard protocol. Any agent that speaks HTTP can use it. There is also an MCP adapter so Claude and Cursor auto discover the tools. The agent sends a query and gets back a ready to inject context string. It never needs to understand the internals.

Why Local First Matters

The default embedding model runs on your machine. No API keys. No network calls. No cost. No data leaving your computer.

I used all-MiniLM-L6-v2 for the prototype. Apache 2.0 licensed. 384 dimensions. 80 megabytes. Runs on CPU. The vectors are four times smaller than OpenAI's, which means four times faster search and 75 percent less storage.

If you want OpenAI, you switch one config line. But the default is open source and free.

How Retrieval Works

When an agent asks a question, this is what happens.

The query gets embedded into a 384 dimension vector. PostgreSQL with pgvector runs an approximate nearest neighbor search to find the 36 most similar memories. This takes about 3 milliseconds.

Those 36 candidates get scored. Each one receives a composite score based on the four signals I mentioned. The top 12 move forward. The rest are dropped.

Then graph traversal kicks in. If a top ranked memory has relationships with other memories, those get pulled in too. A memory about PostgreSQL might surface a related memory about database migrations. The depth and relationship weight control how strongly these connections influence the final ranking.

Everything gets assembled into a single context string. The agent puts this directly in its system prompt. It never sees the scores, the graph, or the database. Just the relevant facts.

First query takes about 26 seconds. Second query, with Redis caching, takes 14 milliseconds. About 1800 times faster.

Five Background Workers

The prototype runs five background jobs continuously.

One generates embeddings for new memories every 10 seconds. Another flushes access counters from Redis to PostgreSQL every minute. A third recomputes recency scores hourly using exponential decay. A fourth archives expired session memories. And a fifth decays confidence for untouched memories every 30 days, eventually marking them as stale.

All of this happens without any manual work. You write memories. The system maintains them.

What Happens When Agents Disagree

This is where it gets interesting. Two agents write conflicting facts about the same thing.

Claude says you prefer Go. ChatGPT says you switched to Rust.

OpenMemory detects this because both memories share the same key. It applies three rules. If the new memory has an explicit source (you directly stated it), it always wins. If the new confidence is 0.2 higher than the old one, the old gets superseded. Otherwise, both stay active but the old one's confidence drops by 20 percent.

The old memory is never deleted. It is marked as superseded and preserved in the event log. Full history. Full audit trail.

The Demo

The prototype includes a demo showing two simulated agents sharing one memory system.

Agent A, acting like Claude, writes project knowledge. Agent B, acting like Cursor, retrieves it for coding tasks. Then Agent A contradicts itself by switching preferred languages. The system resolves the conflict automatically. The old Go preference gets superseded by the new Rust preference.

The demo also shows keyword search, status lifecycle, and the cache speed difference between cold and warm queries. It optionally uses LM Studio for AI generated content.

What This Proves

Building an agent without memory is like hiring a developer who forgets everything at the end of every day. You would fire that person. Yet we accept this from AI agents.

The prototype shows that the core infrastructure is not that complicated. PostgreSQL with pgvector handles the vector search. Redis handles the caching. Five async Python workers handle the maintenance. A standard REST API and MCP adapter handle the integration.

The hardest parts were the contradiction resolution rules and the retrieval scoring weights. Getting four signals to work together properly took several iterations. But the result is a system where agents can share context without ever touching each other's memory internals.

Try The Prototype

Everything is open source under Apache 2.0.

git clone https://github.com/AzazAhmedLipu79/openmemory
cd openmemory
docker compose up -d
docker compose exec api alembic upgrade head
curl localhost:8000/health

Ten API endpoints. Four retrieval modes. Five background workers. Twelve tests. Twelve technical documents. It is a prototype, but it works end to end.

The goal is to prove that shared agent memory should be a standard, not an afterthought. Every agent deserves access to the full picture.

← Back to Articles