AI agents forget. Every time a coding assistant loses track of a debugging thread or a data analysis agent re-ingests the same context it already processed, the team pays in latency, token costs, and brittle workflows. The solution most teams are looking for (expanding the context window or adding more RAGs) is increasingly expensive and still doesn’t work reliably.
To address this, researchers from Mind Lab and several universities proposed delta-meman efficient technique that compresses historical model information into a dynamically updated matrix without changing the model itself. The resulting module adds only 0.12% of the backbone model parameters (compared to 76.40% for a leading alternative) and outperforms it in memory-intensive benchmarks. Delta-mem allows models to continuously accumulate and reuse historical data, reducing reliance on massive context windows or complex external retrieval modules for behavioral continuity.
The long memory challenge
The conventional solution is to simply dump all the information into the model’s context window.
But as Jingdi Lei, co-author of the paper, told VentureBeat, current systems treat memory simply as a context management problem. “Either we continue to expand the contextual window or we retrieve more documents through RAG,” Lei explained. “These approaches are useful and will continue to be important, but they become increasingly expensive and fragile when agents need to operate in long-duration, multi-step interactions, and they don’t really (work) like human memory, as they are more like searching for documents.”
In enterprise environments, the bottleneck is not only whether the model can access history, but also whether it can reuse it efficiently, continuously, and with low latency. Standard attention mechanisms incur a quadratic computational cost as sequence length increases. Furthermore, expanding the context window does not guarantee that the model will actually remember the information effectively. Models often suffer from context degradation or context rot as they become overwhelmed with more (and often contradictory) information, even if they theoretically support a million tokens.
Researchers advocate for advanced memory mechanisms that can represent historical information compactly and maintain it dynamically across interactions. Existing solutions entail important trade-offs and generally fall into three paradigms:
-
Textual memory: stores history as context-injected text, restricted by window boundaries and prone to loss of information under compression.
-
Outer channel (RAG): encodes and retrieves from external modules: adds latency, integration complexity, and possible misalignment with the backbone.
-
Parametric: encodes memory into model weights via adapters: static after training, cannot adapt to new information during live interactions.
Inside delta-mem
To achieve compact, dynamically updated memory, delta-mem compresses an agent’s past interactions into an “online state of associative memory” (OSAM). This state is maintained as a fixed-size array that preserves historical information while the underlying language model remains frozen.
For enterprise workflows, this translates directly to resolving operational bottlenecks. Lei noted that a persistent coding assistant, for example, “may need to remember project conventions, recent debugging steps, user preferences, or intermediate decisions along a workflow.” Similarly, a data analytics agent might “need to maintain task state, assumptions, and previous observations while iterating over multiple tool calls.”
Rather than repeatedly retrieving and reinserting all history relevant to these tasks, the delta-mem array provides a low-cost way to carry forward useful interaction states within the direct model computation.
During generation, the system does not retrieve plain text segments to add to the message. Instead, the current hidden state of the trunk LLM is projected to the array to recover the old memory. This operation extracts context-relevant associative memory signals from delta-mem. These signals are then transformed into numerical corrections that are applied to the model calculations. This directs the model’s reasoning at inference time without altering its internal parameters.
After each interaction, delta-mem updates the online state using “delta rule learning”. When new information arrives, the previous state makes a prediction about the resulting attention values. It then compares this prediction to the actual value and corrects the memory array based on the discrepancy.
This update mechanism is based on a “closed delta rule”. Basically, the memory module has different knobs that control how much old memory is retained and how much new memory is applied. This error correction with controlled forgetting allows the matrix to evolve over time, maintaining stable historical associations without being derailed by short-term noise.
The researchers explored three strategies to determine when and how the matrix is updated:
-
Write to token state Captures detailed changes but is vulnerable to short-term noise.
-
Sequence status write averages the tokens within a message segment, smoothing out updates at the cost of some localized details.
-
Multistate write It breaks memory down into substates for different types of information, such as facts or task progress.
Delta-mem in action
The researchers evaluated delta-mem on three LLM pillars: Qwen3-8B, Qwen3-4B-Instruct, and SmolLM3-3B. They configured the framework with a compact 8×8 matrix. The system was tested on general capability benchmarks including HotpotQA, GPQA-Diamond, and IFEval. It was also tested on memory-intensive tasks, such as LoCoMo, which tests long-term conversational memory, and Memory Agent Bench, which tests retention, retrieval, selective forgetting, and learning at the time of testing in prolonged interactions.
The framework was compared with representative models of three existing memory paradigms: textual memory baselines (e.g., BM25 RAG, LLMLingua-2, and MemoryBank), parametric systems (Context2LoRA and MemGen), and the MLP Memory external channel approach.
Overall, delta-mem outperformed baselines, according to the researchers. On the Qwen3-4B-Instruct backbone, the tokenized write variant achieved an average score of 51.66%, easily beating the frozen vanilla backbone at 46.79% and the strongest baseline, Context2LoRA, at 44.90%. On the memory-heavy Memory Agent Bench, the average score jumped from 29.54% to 38.85%. Performance on the specific learning subtask at the time of the test almost doubled from 26.14 to 50.50.
However, the most compelling conclusions are the operational efficiency of the system. The researchers tested the framework in a context-free environment where the historical text was completely removed from context. Even without explicit text playback, delta-mem successfully recovered context-relevant evidence in multi-hop tasks. The researchers argue that the model remembers past interactions without needing to ingest massive amounts of fast tokens.
The framework also adds only 4.87 million trainable parameters, which represents only 0.12% of the Qwen3-4B-Instruct backbone. In comparison, the MLP memory baseline required 3 billion parameters, scaling up to 76.40% of the backbone size while yielding inferior results. When request length increased to 32,000 tokens during inference testing, the framework maintained almost exactly the same GPU memory footprint as a standard model without modifications. Avoids the large memory overhead that plagues other advanced memory systems such as MemGen and MLP Memory.
Different upgrade strategies were beneficial depending on the capability of the underlying model. The stream state write strategy was most effective for stronger backbones like Qwen3-8B. These more capable models use segment-level writing to smooth updates and mitigate token-level noise. In contrast, the multi-state write strategy drove huge performance gains for smaller backbones like SmolLM3-3B. For these lower-capacity models, separating memory into multiple states was critical to minimizing information interference.
Implementing delta-mem in the enterprise stack
The researchers have published the code for delta-mem on GitHub and the weights for your trained adapters in Hug the face. For AI engineering teams looking to integrate this framework into their existing inference stack, the process requires minimal computing resources.
“In practice, an engineering team would start from an existing instruction-tuned backbone, connect Delta-Mem adapter modules to selected attention layers, train only the adapter parameters on domain-relevant long-context or multi-turn data… and then run inference with the memory state updated online during the interaction,” Lei said. Fundamentally, teams don’t need a massive pre-training corpus. Training data only needs to reflect target memory behavior, such as multi-turn dialogues, agent traces, or domain workflows where prior information needs to influence subsequent decisions.
While compressing interaction history into a fixed-size mathematical matrix creates immense efficiency, it comes with trade-offs. Delta-mem is not a lossless replacement for explicit text logging or document retrieval. Because different pieces of information compete within the same limited state, there is a risk of memories becoming mixed up.
“Delta-Mem is useful when the system needs fast, online, and continuously updated behavioral state,” Lei said. “RAG is best when the system needs accurate recall of facts, citations, compliance, auditability, or access to a large external knowledge base.” Remembering a user’s work style or a multi-step reasoning path is perfect for delta-mem, while retrieving a legal contract or medical directive should remain in a vector database.
This means that the most realistic enterprise architecture in the future is a hybrid approach. Delta-mem acts as a lightweight internal working memory, reducing the need to fetch or replay everything all the time, while RAG serves as a high-capacity explicit memory layer.
“Looking ahead, I don’t think vector databases will become obsolete,” Lei said. “Instead, I expect enterprise AI stacks to have more layers. We’ll probably see short-term working memory within the model, explicit long-term memory in retrieval systems, and policy or audit layers that decide what should be stored, retrieved, forgotten, or exposed to the user.”
