How Stage Whisper Solves the 'Omniscient NPC' Problem: Context Isolation Architecture

ai stage-whisper memory ttrpg

NPCs that can quote something my character said three sessions ago was pretty cool to see. The memory system was working. NPCs were referencing past events, quoting things I said. BUT, it also started a different problem: NPCs started getting omniscient

Here's an excerpt from a recent session, the relevant part is highlighted:

"So, she says, her voice carrying easily over the rustle of leaves, what's the real reason you came looking for me? Curiosity, or are you just trying to avoid unpacking the rest of that chaotic desk of yours?"

That was referencing something that happened in a completely different scene that this character wasn't part of. The character should have had no way of knowing.

Small inconsistencies like this are immersion-breaking. But in other cases, it can be game-breaking. For example, I have a campaign where my character is a secret vampire navigating a world of unsuspecting humans. NPCs started casually hinting they knew my secret, which suddenly made it very difficult for me to get through the day without being suspected. Or in other examples, if I stole an artifact during a heist, player was the one who stole the artifact during a heist, and then an unrelated character makies suspicious comments.

The AI basically decided to use any and all secrets available in the context for dramatic effect, which should really only happen very occasionally for plot reasons.

So why can't I just tell the AI not to do this?

The problem is that you're trying to achieve a negative. "Do not use this information." That's a fundamentally different and much harder task than the positive case: do use the information that's there. In order to train a model to ignore relevant context, you'd need thousands of examples of situations where the information was clearly available but the output deliberately didn't use it. That's almost the antithesis of what an AI is designed to do. For most tasks, ignoring useful context would be failing at the job.

Conceivably a model could be trained with more examples of not acting on secrets. But narrative and roleplay hasn't been a big focus for model creators thus far, so this problem hasn't gotten the attention it deserves.

I realized I had to fix this on that secret vampire campaign. The campaign was set up so my character could pass as a regular human most of the time, and the whole point was navigating that tension without getting caught. It would be a big narrative moment when any NPC found out the secret, because then they'd need to be persuaded to keep silent about it.

What I discovered was that pretty much every potential antagonist somehow knew. Even though I hadn't revealed anything. And that was immediately plot-breaking. It felt as if there was a big sign over my head announcing that I was a vampire. My character was constantly doing damage control, or feeling like every character had leverage over mine.

I experimented with prompts for a long time. Eventually I realized I'd have to just not include the information in the context at all.

The solution, then, is context isolation. If the AI can't help using anything it sees, then the information genuinely has to not be there. This is achievable in Stage Whisper because each scene and memory is already tagged with the NPCs that appear in it, so we can easily filter to only include the characters who were actually there.

But here's the catch. Some secrets are major plot drivers. You can't just delete them entirely, because the story still needs to account for them. Whether it's a heist, or a secret vampire, those secrets are core to the narrative, and it wouldn't be the same story without them.

So we had to split our text generation in two.

The first part I call the Story Director. It directs the overall story: the plot, the narrative beats. It has all the secrets, and can decide how best to spin the story given those secrets. It understands the full picture, including the things no character should know yet.

The second part I call the Scene Director. It "acts out" the scene handed to it by the Story Director. It has access to all the relevant NPC memories and personalities, so it has enough information to play all the NPCs and provide some interstitial narration, but it genuinely doesn't know the secrets.

Some information needs to pass from one to the other, so they can work on the same story without working against each other.

The Story Director produces a synopsis that tells the Scene Director everything it needs to know. It looks something like this:

<synopsis>
  Elian returns to Room 314 with Maeve, the familiar privacy wards offering brief respite from the attention gathered during their spectacular display. Sensing Maeve's need to study after an unusual day, Elian decides to leave their room for some solitary exploration
  <npc_list>maeve-corwin</npc_list>
  <location>room-314</location>
  <new_scene/>
</synopsis>

This tells the Scene Director what's going on in the scene, where it is, who's in it, and whether it's a new scene. The player character's presence is assumed. The Story Director can also optionally emit out-of-character talk to the player, and make detailed performance notes for its own future use.

The two directors don't see the same context. Here's the split:

Story Director sees:

This is everything the Story Director needs to make interesting, consistent plot decisions. It's a lot of data, but the KV cache keeps costs manageable.

Scene Director sees:

This is a deliberately reduced view. The Scene Director has everything it needs to run the scene, and nothing it shouldn't have.

Two character sheets

The public versus full player character sheet distinction is key. The Story Director needs to know the full details and all secrets to produce the story. But the Scene Director should not know those details. It still needs to know who the player character is physically, what they look like, their public reputation, the things any NPC would reasonably pick up just from being around them. Maintaining a separate public sheet means the Scene Director always has that context without ever needing access to the secrets.

Filtering memories by scene

Each scene is tagged with which NPCs are present. This means we know exactly which recent scenes are relevant to each NPC, and we add those to the Scene Director's context. The assumption is that scenes within the current session are highly relevant. For older memories, semantic search surfaces the ones that might matter.

The result is a Scene Director that genuinely cannot know things the characters in the scene don't know. Not because it was told not to, but because that information physically isn't there.

I've been running a campaign where my character is a human female who has to dress as a guy to secure her family's title as minor nobility, due to the laws of inheritance. It's a light-hearted comedy campaign, and it features a few comedy scenarios that come from that gender confusion.

Without this system in place, there's generally no way to surprise NPCs. They all behave like they suspect something. Maybe not directly, but the way they talk to you always has this edge of knowing. It's hard to describe, but you can feel it. The dramatic tension of a secret identity just isn't there.

But with this system in place, the NPCs play it completely straight. They have no idea. And the scenes where the big reveal happens really land, because up to that point there's absolutely no allusion to it. The comedy works because the audience, the NPCs, genuinely don't see it coming.

There is one edge case. Because the Scene Director doesn't know the exact secret, it occasionally makes anatomically incorrect details. This is hard to avoid, since the Scene Director is working from a public character sheet that doesn't include those details. The workaround would be adding a third step: the Story Director reviews the generated scene before it goes out, to catch any discrepancies like this. That adds cost and latency, so it's on the roadmap but not implemented yet.

One thing that did surprise me: when a secret does get revealed in-game, it propagates naturally. The Scene Director holds the previously generated scenes, so once an NPC learns something in a scene, that becomes part of their context for the next turn. At the end of scenes, those scenes form memories for the NPC. At the end of sessions, NPC sheets get updated. Both of these mean an individual NPC's knowledge of a secret gets recorded, and the story can continue with that NPC now genuinely in the know.

The tradeoff

This approach isn't free. Cost increases, though not quite double, since the Scene Director's context is relatively light. The prompt cache helps significantly too. As the scene plays out, new turns get added to the end of the context, which means 90% of the prompt can come from cache rather than regenerated fresh each time.

Currently it's enabled for all campaigns. The architectural changes are significant enough that it doesn't make sense to provide a path that bypasses this setup. For most campaigns the tradeoff is worth it, even if your story doesn't have major secrets, because NPC consistency matters regardless.

What this unlocks: the Game Director

The two-stage pipeline demonstrates something useful: splitting tasks between more specialized AIs with optimized contexts gives better results than one AI doing everything. That's leading to a natural third stage.

I call it the Game Director. Its job is to handle the TTRPG mechanics: dice rolls, skill checks, status tracking. Its context can be optimized differently from the others, since it cares less about narrative history and more about game rules and game state.

Adding a Game Director has an unexpected benefit. Since it runs every turn in case the player tries something that warrants a skill check, it can also decide whether the player input is a continuation of the last scene or something new. If the player is just chatting with an NPC without moving the plot, the Game Director can skip the Story Director entirely and let the Scene Director continue the scene on its own. The Scene Director has the NPC sheets and can do memory searches, so it can carry on dialog like this without the overhead of a full Story Director call.

This is the direction the architecture is going: a three-stage pipeline that gets progressively more specialized, with cost-saving optimizations built in at every layer.

Stage Whisper is built for people who want to play solo TTRPGs with the depth and consistency of a long-running campaign. The memory system, including this context isolation architecture, is what makes that possible. Get started or join our Discord to see how it works in practice.

For more on Stage Whisper's memory system, see Part 1: Narrative Consistency and Part 3: NPC Dreaming.