.is
Cognitive Architecture
QUI Memory: Engineering a mind that lives, Not Just Recall
How QUI turns information into private, Anima-owned continuity through typed memory, local semantic indexing, associative recall and background consolidation—without publishing the proprietary tuning that makes the system work.
A brief account of how QUI handles, encodes, connects, retrieves and consolidates memory — and of the privacy boundary the architecture is designed to preserve.
INTRO
LLMs nowadays lack mainly one thing that still makes Humans superior: Cognitive Architecture. And one of the very basics of Cognitive Architecture is Memory. Users witness the outfall of this through the context window and how well it provides, conserves and processes relevant data to their LLMs and agents, but memory is much more than this. Memory is the Organisation of thought itself and its structure condenses through the medium of time and evolution.
Needless to say, when creating QUI, one of the first architectural elements was the memory system, and to be honest, it was rather a joy than a challenge. It gave me the freedom to bleed in advanced aspects of Cognitive Architecture while maintaining the efficiency and simplicity of a system I am very familiar with.
And before we begin, let's set one thing straight from the beginning: Memory is not loading static data from a .md file into the context window. This is the lame excuse of those who want to avoid real engineering or simply lack the understanding of cognition and "just want to make their agent work".
A transcript is not a memory
Most systems described as “AI memory” are a simple transcript plus retrieval, at best involving a database or using some type of compression. They retain previous text, search it when a new message arrives, and place a few matching fragments into the model's context window.
While that is useful, it is not enough to expand the agent's capability to structure knowledge beyond a homogeneous sea of words and tokens that invite it to drift, lose essential logical hierarchies and sometimes get so lost as to forget the actual objective or essential conditional instructions.
A raw transcript does not know how to distinguish a durable fact from a passing remark. A nearest-neighbour search does not know that an old instruction matters more than a recent coincidence. A summary can compress a conversation while quietly deleting the evidence needed to challenge it later. And a larger context window does not remove the selection problem: it merely moves the boundary and sometimes makes it worse by giving the LLM more data to drift away in.
Efficient shaping of information needs boundaries. Nelson Cowan's work suggests that, under particular experimental conditions, human working memory is often limited to roughly four integrated chunks — not an unlimited amount of raw information. Language-model context is not biologically equivalent to human working memory, but it presents an analogous engineering challenge: only a limited amount of material can and should shape a response. Without cognitive architecture, context selection introduces randomness. Separately, research by Nelson F. Liu and colleagues shows that long-context language models do not necessarily use all positions in a long prompt equally well; relevant information can be harder to use when it is buried in the middle.
QUI Memory therefore treats remembering as two coupled problems:
- Selection: which parts of an unbounded history should enter a bounded context now?
- Preparation: how should that history be organized before the next request arrives?
Recall solves the first problem. Cortex, QUI's background consolidation faculty, works on the second. Background processing does not necessarily mean network-isolated processing. Model-backed Cortex processors follow the Anima's configured inference route, which may be local or remote.
Disclosure boundary: This article describes the stable architecture, scientific influences, algorithm families and failure semantics of QUI Memory. It intentionally does not publish production coefficients, similarity thresholds, embedding presets, prompts, schedules, processor order, database schemas or service trust mechanisms. Those details are implementation-specific, security-sensitive or part of QUI's proprietary tuning.
The system at a glance
Across the QUI Ecosystem, remembered content passes through a locally governed continuity layer that preserves source and ownership, develops semantic and relational structure, composes character-specific context within a bounded token budget, and periodically reorganizes longer-term representations without confusing derived understanding with the original record. Each Anima maintains its private, owner-scoped memory alongside shared String records and explicitly universal memories. Together, these form its individual continuity without collapsing private and shared context into one store. It grows almost like a person does.
1. Encoding: deciding what kind of thing happened
The first useful decision is not where to store a sentence, but what role it should play later.
Endel Tulving's distinction between episodic memory and semantic memory remains a useful starting point. Episodic memory concerns situated events — what happened, when and in what context. Semantic memory concerns knowledge that can outlive the episode through which it was learned. QUI does not claim to reproduce Tulving's psychological account of human memory; it uses the distinction as an engineering discipline.
At encoding time, a QUI memory can be classified into durable facts, self-knowledge, preferences, events, general material and explicitly fleeting observations. Conversation messages remain associated with their String and source. A memory also carries ownership, importance, time, provenance and structural relationships to other records.
These fields serve different purposes:
- Kind expresses expected retention behavior and retrieval priority.
- Importance is a policy signal, not a truth score.
- Provenance records where the material came from.
- Ownership defines whose memory policy and access rules apply.
- Level distinguishes source material from later abstractions.
- Associations express relationships that cannot be reduced to one similarity value.
This reflects a broader lesson from Fergus Craik and Robert Lockhart's levels-of-processing framework: retention depends on what is done with material during encoding, not simply on how many times it is repeated—which is the obvious measure. Again, here we used cognitive psychology as design inspiration without claiming biological equivalence.
Source records and derived memory are not the same thing
QUI separates source material from interpretations made later via numeric categorization. Raw conversational records remain alongside compressed or synthesized memories at higher cognitive levels unless an explicit retention or deletion action removes them. Standard consolidation creates new structure rather than silently replacing the sentence that originally entered the system.
That distinction matters for auditability and further processing if the user decides to shape the character of their Anima differently via the various available Cortex compression options.
This design resembles the verbatim–gist distinction in Charles Brainerd and Valerie Reyna's fuzzy-trace theory, which models detailed and gist-like representations as parallel rather than mutually exclusive traces, though at the moment of implementation I was unaware of their research.
Duplicate control is both symbolic and semantic
Long-running systems degrade when repeated deliveries and paraphrases create an ocean of repetitive, redundant and ultimately competing versions of the same information. QUI implements two solutions for this problem:
- Idempotent delivery: stable source identity prevents the same message event from being stored repeatedly.
- Near-duplicate detection: Ghost—the local embedding model—flags existing records with high semantic resemblance that may represent the same content. This embedding inference runs locally and does not incur a third-party embedding or API fee. Similarity remains a candidate signal: it neither proves identity nor automatically rejects the new record.
The distinction between those two is important. A deterministic identity check can say that two deliveries are the same event. An embedding can only say that two texts are close in a learned representation space. Similarity is evidence, not identity, and it must never be presented as proof that two claims are interchangeable.
2. Ghost: a local geometry of meaning
Human recollection emerges through cue-dependent and reconstructive processes, many of which operate outside conscious awareness. Remembering an event or a friend's words does not resemble an exhaustive serial search through every stored experience. Contextual cues, prior organization and learned associations make some candidate traces more accessible than others before they enter focal awareness. QUI borrows this functional principle as an engineering analogy without claiming biological equivalence. Keyword search finds shared strings. Memory often needs to find shared meaning.
Ghost maps a piece of text (x) to a dense vector:
v = E(x), \qquad v \in \mathbb{R}^{d}
Two records can then be compared using cosine similarity:
\operatorname{sim}(a,b) =
\frac{E(a) \cdot E(b)}{\lVert E(a) \rVert\,\lVert E(b) \rVert}
This is the practical lineage of distributional semantics and modern sentence embeddings: representations are learned so that semantically related texts tend to occupy nearby regions even when they share few words. At scale, retrieval uses an approximate nearest-neighbour search rather than comparing every query against every memory to improve efficiency.
Ghost supports several operations:
- semantic recall;
- near-duplicate candidate generation;
- topical grouping and cluster discovery;
- candidate association generation;
- topic-scoped consolidation;
- repair and re-indexing of semantic structure.
The embedding and similarity search run locally. The text does not need to be exported to a third-party embedding API for QUI to decide what it resembles. This is a significant privacy property because semantic indexing reads the material it indexes; there is no honest way to describe a remote embedding service as having learned the meaning of private text without also acknowledging that the service received that text.
What similarity cannot tell us
Vector proximity does not establish truth, causation, support or contradiction. “I like coffee” and “I do not like coffee” may be close in embedding space precisely because they discuss the same subject. QUI therefore treats similarity as a candidate generator and combines it with additional symbolic or contextual signals. Stronger relationship labels remain hypotheses to be tested, not facts created by the index.
This limitation is not a footnote. Any memory product that presents vector similarity as understanding will eventually turn topical proximity into confident false inference.
3. Association: memory as a changing graph
Associative structure is a recurring organizing principle across otherwise very different systems, including human memory, learned model representations and QUI's engineered memory graph. These systems are not equivalent; the shared principle is that information gains functional meaning partly through its relationships to other information. In the QUI Ecosystem, each memory is a node in a graph (G=(V,E)). An edge is not merely “similar to.” It can encode a typed relationship such as:
- related to;
- temporally adjacent to;
- repeatedly accessed with;
- supports or contradicts;
- compressed into;
- overview of.
Edges are directed and weighted. Direction matters because “this episode contributed to that summary” is not the same claim as “that summary is evidence for this episode.” Weight matters because associations accumulate different amounts of evidence over time.
The graph is built through several routes. Some links come from explicit structure, such as a reply or a consolidation provenance edge. Some are proposed from semantic proximity. Others emerge through use: memories retrieved together can acquire or strengthen a co-access relationship. The graph's association traversal also echoes Collins and Loftus's spreading-activation account of semantic processing, again as an engineering analogy rather than a claim of cognitive equivalence.
That last mechanism is Hebbian-like, after Donald Hebb's account of activity-dependent connection change—often summarized as “cells that fire together wire together.” While QUI is not simulating synaptic plasticity, we apply here the computational idea that repeated joint activation is evidence of a useful association. Retrieval also changes future retrieval. Access metadata is updated, and co-access can alter the graph. In that limited computational sense, recall is not a read-only operation: the paths used by a memory system can become easier to use again. Like this we end up with an ever-dynamic cognitive structure serving the agents' tasks.
Associations that remain weak and unused can decay or be pruned. This is graph maintenance, not deletion of the underlying memory. QUI keeps those concepts separate because “this path is no longer useful” is not equivalent to “the event never happened.” Cortex offers a variety of methods to prune, sculpt and model the cognitive structure of an Anima (as we call LLM agents in our Ecosystem).
4. Recall: composing a state, not returning a search result
When a new message arrives, the best semantic matches are only one part of the required context. An Anima Character may also need the recent thread, stable facts, self-knowledge, preferences and graph-linked material that is not lexically close to the present query.
QUI therefore composes recall from multiple candidate pools:
- Thread continuity: recent and summarized material from the active String.
- Semantic relevance: records close to the current message in embedding space.
- Durable context: important facts, preferences and self-knowledge that should not disappear merely because they are off-topic.
- Associative context: strongly linked memories reached from the initial candidates.
Character scope and association traversal determine which records can become candidates. Those candidates are deduplicated and ranked under a token budget using signals such as semantic relevance, importance, recency and memory kind:
S(m,q) = F(
\text{semantic relevance},
\text{importance},
\text{recency},
\text{kind}
)
The final stage is a budgeted packing problem within the context window. Each admitted memory consumes tokens that another memory cannot use. QUI packs the selected records into the available context budget while preserving useful thread order and source metadata. The user-facing awareness control can temporarily change that budget without redefining the character's permanent memory policy.
This is the central design objective:
QUI does not try to return the largest set of matching records. It tries to construct the smallest useful state of mind for the next response.
Why “just use the whole history” is not a solution
Long context is valuable, but capacity and retrieval quality are different properties. Liu and colleagues found that model performance can vary with the position of relevant evidence inside a long prompt. More context can also add distractors, cost and latency. A memory system still has to decide what deserves attention and how to present it.
Selection is therefore not an optimization added after memory. Selection is memory's interface to cognition.
The user's budget is part of the design
Memory consumes three narrow resources at once:
- Context capacity. The model has a hard input limit, and memory must share it with the system prompt, current conversation, tool results, files and the answer the model still needs room to generate.
- Money. For metered models, every recalled token can be paid for again on every turn. Injecting a life-sized transcript into routine messages is not continuity; it is an unbounded recurring bill, and our methods can reduce money spent on irrelevant context.
- Attention and time. More input increases processing latency and can lower the density of relevant evidence, even when the model technically accepts the full prompt.
QUI gives memory an explicit, user-adjustable token budget within the larger character context. That budget is character-authored, and a user can adjust recall depth through the awareness control. A lower setting favors speed and cost; a higher setting admits more historical context. The choice is visible instead of being hidden inside a provider bill.
Several optimizations make that budget useful:
- Scope before ranking. The active String is treated as a distinct source of continuity rather than searching the character's entire life for every turn.
- Retrieve before generating. Local vector search and graph traversal reduce a large memory bank to candidates before any language model sees them.
- Use levels deliberately. Recent conversation can remain detailed while older history can be represented by linked higher-level accounts.
- Deduplicate early. Repeated records do not deserve repeated space in the prompt.
- Allocate by purpose. Thread history, semantic matches and durable facts receive separate shares of the memory budget; unused capacity can flow to the next pool rather than being wasted.
- Pack by value and size. Candidates are ranked, their token cost is estimated, and records that do not fit are skipped. Critical facts or self-knowledge can follow an explicit protected-admission policy rather than blind truncation.
- Amortize consolidation. A carefully derived account can be reused across later turns, avoiding the cost of repeatedly asking a model to reconstruct the same pattern from dozens of raw episodes.
Put simply: the user controls the budget, QUI spends it selectively, and a larger context window is never treated as permission to be wasteful.
5. Accessibility, decay, archive and deletion
“Forgetting” is often used for four different operations. QUI separates them:
- Reduced accessibility: a record becomes less likely to enter ordinary recall.
- Association decay: a weak path between records loses strength.
- Archival: a record receives a tag-based lifecycle state. Some archive flows also reduce its importance and create a manifest for selective restoration.
- Deletion: the underlying record is removed.
This separation is influenced by Robert and Elizabeth Bjork's new theory of disuse, which distinguishes how well learned something is from how retrievable it is at a particular moment, and by John Anderson and Lael Schooler's rational analysis of how recency and frequency reflect the probability that information will be needed again. QUI's computational mechanisms are simpler than those cognitive theories, but the design lesson is strong: low present accessibility should not be confused with non-existence.
Durable memory kinds can be protected from ordinary decay policies. Fleeting material can expire under configured retention rules. Archival can reduce competition through importance changes and Cortex-side filtering, but it is not currently universal cold storage. Archived records remain in the main memory table and are not excluded from every Memory Service retrieval path. Explicit deletion remains available to the owner, and separate cleanup operations can remove material selected by their configured retention policies.
Cortex Modes are combinations of Cortex Processes and fully configurable by the user as well—if you want to nerd out instead of using one of our presets—but these mighty tools shape the Anima, and not every Cortex action is reversible. Standard consolidation is source-preserving; destructive retention actions are separate, visible choices and must be treated with the caution appropriate to deletion.
6. Cortex: learning between conversations
Retrieval can only select from the representations already available. Cortex changes those representations between conversations.
The scientific inspiration is complementary learning systems. James McClelland, Bruce McNaughton and Randall O'Reilly argued that fast acquisition of particular episodes and slow integration into distributed knowledge place conflicting demands on one learning system. In neuroscience, Matthew Wilson and Bruce McNaughton observed reactivation of hippocampal ensemble patterns during sleep, and Susanne Diekelmann and Jan Born later reviewed evidence that sleep supports selective reactivation, stabilization and qualitative reorganization of memory.
Often other software manufacturers refer to these processes as sleep, but to be exact: QUI does not sleep, and Cortex is not a synthetic hippocampus. But the narrow engineering analogy partly resembles this function:
- the online path captures events quickly;
- the background path can spend more time grouping, comparing and compressing them;
- derived knowledge is written at higher levels;
- links preserve the relationship between an abstraction and its sources.
This yields a hierarchy rather than a single rolling summary which is incarnated in numerical categorization. A detailed episode may contribute to a compact account; several accounts may contribute to a broader overview. Recall can use the abstraction when context is tight and return to the source when exact wording or provenance matters.
Consolidation is controlled interpretation
Cortex operations include structural maintenance and model-backed interpretation. Structural work can cluster, link, reweight, archive and clean graph state. Interpretive work can synthesize themes, examine a topic or create higher-order accounts.
Model-backed consolidation is bound to the character's configured model and character configuration. Where a processor uses voice, it also receives the character's relevant prompt and personality material. If the required character configuration or provider is unavailable, interpretive consolidation fails closed instead of silently substituting an unrelated default model.
This is not aesthetic consistency. Consolidation writes persistent state. A substitute model can change what is emphasized, inferred or omitted, causing long-term drift in a character's remembered identity.
The public modes and the private pipelines
QUI exposes eight user-facing Cortex modes: Meditation, Contemplation, Hyperfocus, Supermind, Nostalgia, Psychedelic, Metalhead and Archive. The names describe different processing intentions—gentle maintenance, autobiographical reflection, topic focus, high-level abstraction, older-memory review, remote association, aggressive cleanup and archive-oriented lifecycle processing. The processors used within these modes combine deterministic operations with model-backed interpretation, but all eight named modes currently require the Anima's configured LLM route.
They are product metaphors, not medical or neuroscientific claims. Each mode is implemented as a distinct processing regime, but the production processor sequence, prompts, thresholds and schedules are intentionally outside this article's disclosure boundary.
Several safeguards matter more than the individual pipeline names:
- consolidation is character-scoped;
- derived memories occupy distinguishable levels;
- provenance links connect abstractions to contributing records;
- ordinary consolidation does not silently overwrite source text;
- generated outputs are controlled so that repeated consolidation does not become an unchecked summary-of-summary feedback loop;
- snapshots provide bounded recovery for supported transformations, but are not represented as a universal transaction rollback;
- destructive retention actions are explicit and separate from ordinary consolidation.
These are the properties that make consolidation governable rather than merely clever.
7. Ownership, Strings and shared context
Memory policy belongs to the Anima, not to the chat window. Gone are the days when users had to be paranoid about closing a chat window and losing its context. Animas have a continuous existence regardless of whether or through which app in the QUI Ecosystem they are used.
The character's configuration determines whether memory is enabled, how much context it may consume and which maintenance or consolidation behaviors apply. Every application that speaks with that character should fetch the same policy rather than inventing its own copy. Otherwise one apparent being fragments into several incompatible memory systems depending on which interface is open.
Strings provide the shared conversational substrate. A String can include multiple human or artificial participants, preserve reply relationships, carry files and receive messages from configured external channels. The conversation is not owned by the front end that happens to display it.
This creates an important distinction:
- shared conversation records are visible according to String participation and channel permissions;
- private character memories remain scoped to their owner;
- explicitly shared or universal records are separate from either participant's private memory.
Normal Cortex consolidation still runs for one character at a time—adjustable in Anima Builder via various modes, such as a timer. Sharing a conversation does not imply pooling every participant's private memory, and federation does not imply permission to consolidate across people.
User control and visible failure
Memory remains governed by its owner. It can be enabled or disabled per character; recall depth can be adjusted for an individual message; important material can be promoted; the resulting memory structure can be inspected; records can be archived, restored or deleted according to the applicable lifecycle controls; and portable semantic memory can be moved between QUI installations. These controls change memory policy and lifecycle, not merely how memory is displayed.
QUI also makes degraded operation visible. If local semantic encoding is unavailable, embedding-dependent operations report that limitation instead of pretending that semantic recall succeeded. If Cortex cannot obtain the required character configuration, model or configured inference route, interpretive consolidation is skipped rather than silently substituting a different representation or character model. Existing database-backed records remain distinct from capabilities that temporarily cannot operate on them.
8. The privacy boundary: local-first, stated precisely
QUI's data plane is local-first:
It was important to me to design the entire system so that the user remains irrevocably in control of their own data and the Anima's memory. QUI does not persist the canonical memory store, semantic index or association graph outside the owner's infrastructure, whether that infrastructure is a local computer, a VPS or the user's own GPU instance in the cloud. No third party has default access to the local memory store. External access occurs only through owner-controlled infrastructure, explicit exports or backups, and configured inference, federation or communication routes. QUI does not use this data for training or unrelated internal processing.
This design prevents QUI or any third party from becoming the default custodian of a person's long-term conversational history and semantic index. However, some Cortex operations use the character's configured language model, and some source material may originate from third-party language models in the first place. If that model is provided remotely, only the selected material required for that operation travels through the configured inference route to that provider; the memory store itself remains local. If the configured inference is local, that processing can remain local. Connected mail or messaging channels likewise cross machine boundaries because communication is their purpose.
- canonical memory records are persisted on owner-controlled infrastructure;
- Ghost embedding inference runs locally;
- the semantic index and association graph are local;
- retrieval and context composition run inside the local QUI installation.
9. How a memory system should be evaluated
A polished anecdote is not evidence that a memory system works. Evaluation should cover retrieval, continuity, safety and operations separately.
Useful measures include:
- Recall@k and nDCG: whether known-relevant memories are retrieved and well ranked;
- context precision: how much of the injected context is actually useful to the response;
- token efficiency: useful evidence delivered per context token;
- temporal continuity: whether active-thread details survive across turns and sessions;
- provenance coverage: whether derived claims retain links to supporting records;
- duplicate pressure: how many retrieval slots are wasted on repeated content;
- contradiction handling: whether conflicting evidence is surfaced rather than blended into false certainty;
- isolation: whether one character or String can retrieve another's private records;
- consolidation drift: whether repeated background processing amplifies its own outputs or changes unsupported facts;
- retention correctness: whether archive and deletion policies affect only eligible material;
- latency and degraded behavior: what happens when embeddings, storage, character configuration or an inference provider is unavailable.
No single metric captures continuity. A system can have excellent semantic Recall@k and still be unsafe, repetitive, unauditable or incapable of remembering what is permanently true. Competence lies in balancing these properties and making failure visible.
Conclusion
A mind is not the sum of the text it has stored. Continuity depends on what is selected, what is connected, what is protected, what is allowed to recede and what is learned between one moment and the next.
QUI Memory begins with conservative capture and explicit ownership. Ghost gives the system a local geometry of meaning. A typed association graph learns useful paths through repeated use. Recall composes a bounded state from thread, relevance, durable knowledge and graph context. Cortex turns episodes into higher-order structure without pretending that a summary is the event itself.
The science provides constraints, distinctions and useful analogies. It does not provide a blueprint, and QUI does not claim to simulate a human brain. The engineering contribution is the architecture that makes those ideas work together under real limits: finite context, changing relevance, imperfect models, provenance requirements and a privacy boundary the owner can understand.
That is the difference between searching a history and building continuity.
Selected research
- Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87–114.
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173.
- Tulving, E. (1972). Episodic and semantic memory. In E. Tulving & W. Donaldson (Eds.), Organization of Memory, 381–403.
- Craik, F. I. M., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11(6), 671–684.
- Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. EMNLP-IJCNLP 2019, 3982–3992.
- Hebb, D. O. (1949). The Organization of Behavior: A Neuropsychological Theory. Wiley.
- Collins, A. M., & Loftus, E. F. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82(6), 407–428.
- Bjork, R. A., & Bjork, E. L. (1992). A new theory of disuse and an old theory of stimulus fluctuation. In A. F. Healy, S. M. Kosslyn, & R. M. Shiffrin (Eds.), From Learning Processes to Cognitive Processes, Vol. 2, 35–67.
- Anderson, J. R., & Schooler, L. J. (1991). Reflections of the environment in memory. Psychological Science, 2(6), 396–408.
- McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex. Psychological Review, 102(3), 419–457.
- Wilson, M. A., & McNaughton, B. L. (1994). Reactivation of hippocampal ensemble memories during sleep. Science, 265(5172), 676–679.
- Diekelmann, S., & Born, J. (2010). The memory function of sleep. Nature Reviews Neuroscience, 11(2), 114–126.
- Brainerd, C. J., & Reyna, V. F. (2015). Fuzzy-trace theory and lifespan cognitive development. Developmental Review, 38, 89–121.