AI Roleplay Memory Gets Better By Dreaming

One of the first scenes I got into after adding memory recontextualization — what I've been calling NPC dreaming — was a tender moment between my player character and a love interest. I play a character who's conflicted in his relationships for various story reasons. In this scene, he'd just pulled back from the love interest, saying he couldn't commit due to these other conflicting responsibilities.

The character then surprised me by citing not one but two earlier conversations. The first time, she was directly quoting something I had said in dialog — "the real you felt locked away," something I'd offered to explain why my character was conflicted. In the second sentence, she quotes something she had said in an earlier session, and uses it as a moral anchor for the moment we're in right now.

"I know," she says quietly. "About the job. The responsibilities. The others." She doesn't look away. "I was there when you said the real you felt locked away. I've watched you navigate all of it."

She leans closer, her forehead finding yours — returning the gesture you offered earlier, but now as an anchor rather than an escalation. Her breath is warm against your face.

"Do you remember what I told you at dawn?" she asks. "On your couch. That people get hurt when they care. That you don't get to opt out of that by not choosing." Her thumb stills against your hand. "The hurt is already part of the story. It always was."

Just saving memories isn't enough, recall gets fuzzy

Before this new set of changes to the memory system, Stage Whisper was already storing NPC memories for every scene, and the NPC does recall those facts, but they were often fuzzy. Sometimes newer memories were being ignored or mixed up with older ones. Other times the NPC wouldn't recall a specific memory until prompted by the player. The experience was decent, but often lackluster. It was brilliant when it worked, and the NPC pulled out the exact memory expected and responded in character. But often it missed the memory, or got the sequence of events wrong. It didn't know what it didn't know to search for.

Recontextualization also helps keep the sequence of events straight. Where before an NPC sometimes got events mixed up, recontextualized memories store the exact sequence of memories, helping the NPC keep track of what happened.

The most extreme example of this is a character that you are forced to betray at one point, but then later reveal that it was done out of necessity or for a greater purpose. This often sets up two conflicting memories, one of the betrayal, and one of the big reveal. Depending on which memory gets recalled could result in the NPC behaving differently. So previously just having NPCs make memories, and be able to recall them works to an extent, but it can get mixed up, and it doesn't know what it doesn't know to search for.

Memory Recontextualization: how NPCs dream

The solution to the mixup problem is that newer memories need to recontextualize old ones. As NPCs learn new information, they need to be able to adjust their viewpoint, and not just throw a bunch of memories into a bag, and pull them out at random. Older memories need to be superseded and changed by newer ones when the NPC learns the truth about an event and changes their perspective.

In the betrayal case, the betrayal and then discovery both are big parts of the character development. You can't just have one without the other, the story would be incomplete. So a record of both events are needed, and they need to be linked together so that one explains or recontextualizes the other.

This is what I'm calling the recontextualization stage, or in other words "dreaming": when NPCs can sort out their memories, and process the new ones to bring new understanding of old ones. Stage Whisper now does recontextualization at the end of every scene when new memories are made. Old memories are recontextualized, producing recontextualized memories that are fetched in preference to the raw underlying unlinked memories. As NPCs grow, they'll keep building up their memory in this way, making it part of their character development.

Proactive recall: memories that find themselves

Even with recontextualization, NPCs currently have to know to search for something before they need it. This tended to slow down the scene writing and make it more expensive because the AI had to stop what it was doing, go do the memory search, and then write out the scene. To help both with the costs, and the fact that the NPC doesn't consistently reach for its memories, I added a proactive recall step. It's an extra set of steps that can be turned on in the settings (same place that lets you toggle between the one-step, and two-step processing described in a previous blog post), that proactively fetches relevant memories for a given scene, so that when the AI writes the story or the scene, it already has those memories on hand.

With proactive recall, NPCs are much better at referring to past memories, and are consistently aware of those memories, where before they had to stop and search for something they had stored.

What this unlocks

The new proactive memory fetching combined with recontextualization (dreaming), plus the already existing measures in place for context isolation, means that NPCs are a lot better at referring to past memories, and are consistently aware of those memories than before. The narrative is also written more "with intention," with in mind a knowledge of those past events. The effect on narrative is subtle, but you can feel it when a passage is written in reference to past events, versus when it's written without any knowledge of events.

That's what made the scene at the top of this post possible. The NPC wasn't fishing around for something she'd stored. She had it on hand, and she used it at exactly the right moment.

Why this matters for the kind of story you want to build

Accurate and consistently proactive memory recall is extremely important in the kind of long-form storytelling and roleplay that Stage Whisper was built for. When you have long campaigns, you want NPCs to grow with you, to develop memories and knowledge of what you do together, your jokes, your romances, other things. It's what allows the campaign to get better and better over time, rather than playing like a bunch of people with partial amnesia that you have to keep reminding about the good times you've had together. It's particularly good for slow-burn romance, like the example at the top of the post, where characters can quote things you've said directly.

That's a relationship worth building.

For more on Stage Whisper's memory system, see Part 1: Narrative Consistency and Part 2: Context Isolation.

Stage Whisper is built for people who want AI companions and AI GM campaigns where what you say actually matters. Get started or join our Discord to see what memory looks like when it's done properly.