The Memory Gap
In the last dev log, I talked about how important it was for the AI to be able to remember what happens in your stories—the quests you’re on, the locations you discover, the characters you meet, etc. When the AI forgets the choices you’ve made and the characters you’ve met, then in many ways, it breaks the promise of AI Dungeon, of an experience where your choices really matter, and you can shape an open-ended world. Since sharing that blog post, I heard from many of you that much of what I said resonated with your experience. It was clear that this was a big part of what held you back from being able to immerse yourself even deeper into the experience.
Over the past few weeks, I’ve been deep in the code, designing and building the solution for this problem into the new AI Game Engine we’re building for Heroes, and I’m really excited about what we’ve built.
Modeling the Brain
In designing this solution, we took inspiration primarily from how the human brain works. At a broad level, I think the human brain really has two main ways it handles memory for the vast numbers of things we see, hear, and experience every day (and there is a third we developed early in the history of civilization).
Memory Compression
The first is memory compression. We’re constantly taking the things we experience and compressing them into more dense forms. For example, if you closed your eyes right now, it’s unlikely that you could remember every single word you read in the paragraph above. Your brain has already compressed that into a more dense memory, taking less space to store.
Memory Retrieval
The other feature of the brain is its complex memory retrieval system. Have you ever had the experience of recalling something you haven’t thought about in a long time? While you may have many memories stored in your brain, they don’t come to your awareness or current context unless they are relevant to what you’re doing. If I say the word “fire hydrant,” your brain will suddenly bring to mind your memories around it. You might think of its color, shape, function, or any memorable experiences with a fire hydrant you might have had. But it won’t do that until that memory gets triggered.
Stored Structure
Finally, there is a third mechanism we use to remember, but one that we developed as a technology rather than a biological function. This is the ability to read and write text and structured data. This third mechanism lets us record things in perfect detail and do so in specific structures that are useful later. We started with basic symbols, then developed language, and now we finally have all kinds of structures (for example, Excel sheets) that let us record data any way we need.
Heroes Engine Memory
In building the memory system for the Heroes Engine, we built a similar structure. We have three main ways we now store memory: Hierarchical Summarization (Memory Compression), a Memory Bank (Memory Retrieval), and Game State (Stored Structure).
We’ve had the Game State system in Heroes since the beginning, but both the Summarization and Memory Bank systems are completely new.
Hierarchical Summarization
Now, when a player takes an action in Heroes, several things happen. Instead of simply leveraging a window of the previous messages, we instead summarize each message to just the important info the AI needs to remember. A huge amount of the AI’s writing isn’t that important to remember later (do we really need to remember that “the sunlight drifted through the leaves of the forest?”), so we don’t actually need to store that information.
Just like in the memory of a human Dungeon Master, we want to focus on storing and leveraging the summary of the important things that happened, not necessarily each individual word.
But we don’t just do one layer of compression, otherwise we’d eventually run into the same context window issue that we have in current AI Dungeon. Instead, we are continually summarizing early parts of the story into higher and higher compressed forms of memory so that the AI always has the whole story span in its context, at least at a high level (with more recent parts having greater detail in its context).
Because to do great storytelling, you really need to have that high-level context. How could Tolkien write The Return of the King if he isn’t thinking at a high level about what happened in all the other books? He doesn’t have the whole Fellowship of the Ring loaded in his brain at any given time, but he does have the high-level context. And when he needs to, he looks up or remembers specific parts and passages—which brings us to the next system.
Memory Bank Retrieval
If we just had Hierarchical Summarization, while the DM might know the entire history at a high level, in that compression process, it would forget important details that might have shown up earlier in the story. To adapt to that, we also have a Memory Bank where the uncompressed memories are stored to be retrieved when needed.
Whenever memories are compressed to higher forms (and thus removed from the context at some level of detail), we insert them into the Memory Bank. Then, whenever we are generating stories, we can use AI embeddings to find the most relevant memories for any given point in the story. This lets us grab parts of our memory in detail when it’s important. Just like your brain remembering details about a fire hydrant, if you go back to the village of Eldwin’s Hollow, you can pull the most relevant memories of the village, perhaps remembering its appearance, structure, and what happened the last time you were there.
Game State
This has been part of the earliest iterations of Heroes because, to display things like inventory, character stats, etc., we need specific structures that we can feed into the AI. The nice thing is we can also feed those structures back into the AI so that it always remembers the important info we have stored in the Game State: your character’s class and skills, their inventory, their current quests, etc. That way, the AI will never forget those key details of your character and adventure.
What we store in the Game State started small and has expanded over time. We expect that trend to continue as we keep developing Heroes and add new systems that need it.
To Better AI Roleplaying
All of this serves to help make AI Dungeon into the AI RPG that we (all of us at Latitude and in our amazing community) have dreamed of: a world and game where you can be anyone you can imagine, where anything is possible, and your choices truly matter. We’re getting closer and closer. I think you all are going to love Heroes. 😄
I know many of you have asked when we’re releasing Heroes, and while we don’t have a firm release date we can share, I will say it’s getting closer and closer each week.
We’ve finished the core engine and are now building the AI infrastructure that will let us deploy it at a price point that is affordable to as many users as possible.
We really are so, so grateful to all of you for your patience and support. Because of you, we can take the time we need to build something incredible that has never existed before. I’m so excited to get to the point where we can share it with all of you.
– Nick
