Your AI's Memory Should Live Somewhere You Own

Last essay I made a case that the hard problem with AI memory isn't remembering, it's obeying. This one is about the memory itself. Specifically: where it lives, who can read it, and why the answer to both should be you.

Let me start with an uncomfortable question. Every one of these AI memory tools stores your project's context somewhere. Your architecture. Your decisions. The private reasoning your team worked through to arrive at them. So here's the question nobody in the sales pitch wants you to ask out loud: where does all that actually sit, and who else can get to it?

For most tools, the honest answer is "our servers." Which means the most sensitive map of your codebase, the one you built specifically so an AI could understand your project, is now sitting in a database you don't control, governed by a privacy policy you have to trust, deletable only when they decide to delete it. You solved the amnesia problem by handing your project's brain to a stranger.

I did this too, in an early version of what I was building. I stored everything, on my server, so the AI could remember. And it worked. But every conversation I had with a security-conscious developer ended at the same wall: "so your company can read all of this?" And the honest answer was yes. That yes was killing interest in the thing.

The fix was sitting in a tool I already used every day

Here's where it clicked. Developers already have a place where they keep the most important, most private, most carefully versioned information about their work. It's called a git repository. They trust it. They own it. They back it up without thinking about it. It tracks every change with a timestamp and an author. It lets them delete things, branch things, inspect things.

So why was I building a database to store project memory badly, when git already stores exactly that kind of thing well?

The idea is simple enough to say in one sentence. Your AI's memory lives in a private repository that you own. Not a repo full of code. A repo full of memory: the project briefing, the rules, the skills, the record of what worked. It's yours. It sits on your git host, under your account, behind your access controls. And the tool that uses it, mine or anyone's, reads from it and writes to it without ever needing to keep a copy.

Once I saw it that way, a whole pile of hard problems just dissolved.

Versioning? That's git log. Every change to your project's memory is already a commit, timestamped, attributed, diffable. Want to see how your conventions evolved over six months? It's right there in the history.

Backups? Your git host already does that. You don't build a backup system for your memory; you inherit the one you already trust.

Deletion? You delete the repo. When someone asks "can you really remove all my data?", the answer isn't a support ticket and a thirty-day promise. It's: it's your repo, delete it, it's gone.

Audit trail? Inherent. Every rule that changed, every skill that got added, shows who did it and when. You didn't build that. Git gave it to you free.

I had spent real effort building a database that impersonated git. The moment I stopped fighting it and just used the real thing, the design got smaller and better at the same time. That doesn't happen often. When it does, it usually means you found the direction of the wood grain.

The part that makes security people relax

Here's the consequence that matters most, and it's the one I wish I'd had back when that "so you can read all this?" question kept ending my conversations.

When your memory lives in your repo, my servers don't have to hold it. In the strongest setup, a request pulls your memory straight from your git host using your own credentials, my infrastructure never sees it at rest, and when the work is done there's nothing of yours sitting on my side to lose, leak, or be subpoenaed for.

That's not a privacy policy. It's a different shape. A privacy policy is a promise not to look at something you're holding. This is just not holding it. You can't leak what you never had.

I'll be straight about the limits, because pretending they don't exist is exactly the kind of thing that erodes trust. To inject your memory into a live request, something, somewhere, has to assemble the request in plaintext, and whatever assembles it sees it. That's not a flaw you can engineer away; it's physics. What you can do is control who that "something" is. Route it through the AI vendor you already trust and your own git host you already trust, and add no new party. Or run the whole thing on your own hardware and add no outside party at all. The point isn't that plaintext never exists. It's that no new stranger ever gets a copy.

Any git host. That part is not negotiable.

One more thing, and to me it's not a detail, it's the whole spirit of it.

This can't be welded to one git host. The instant your memory only works on GitHub, you've traded one lock-in for another, and you've made a liar out of the word "own." A thing that only lives in one company's product isn't yours in any way that counts.

So it's plain git. GitHub if that's where you live. GitLab, Bitbucket, a self-hosted Gitea box in your own server closet, a bare repository on a drive you can hold in your hand. The format is markdown and commits. It reads and writes the same way against any of them, because underneath the conveniences it's nothing but git doing what git does.

That gives you a spectrum, and you pick your own point on it. Maximum convenience: let a hosted service manage it for you. Maximum control: your own git host, your own machines, memory that never touches anyone else's infrastructure including mine. Same memory, same method, you choose how much you're willing to trust and to whom. Nobody offering you AI memory today lets you make that choice, because the choice is bad for their business. It's good for you, which is the only reason it's in here.

Why I'm telling you instead of just selling you

I could have kept the format secret and made it a proprietary moat. I decided to publish it instead, out in the open, documented, versioned, free for anyone to read or write.

Two reasons. The practical one: the moment your memory is a readable repo in your account, the format is out anyway. You can open your own briefing file and see how it's built. Pretending it's a secret would just add friction without adding secrecy. The honest one: a memory you "own" through a format only I understand isn't really owned. It's rented with extra steps. If I mean the word own, the format has to be public, so your memory outlives my company, my tool, and me.

That's the foundation. Memory that's yours, on infrastructure you control, in a format nobody can take away from you. Next essay I'll get concrete about the part everyone asks about first: how you actually get that memory into every AI session, guaranteed, without installing a thing.

I'm Kevin Rodenhofer. Thirty years making technology behave, now building AI infrastructure under my company OuterData Corporation. If you want your AI's memory to belong to you, get in touch and follow along. The whole method, out in the open.