A returning customer says, “Use my usual delivery arrangement.” Remembering the preference could save a conversation. Remembering a previous customer’s arrangement could create a privacy incident. Remembering an outdated instruction could send an order to the wrong place.
Long-term memory makes an AI assistant more useful only when the application can distinguish a preference, a current fact, an old fact, and an authorization decision. The first implementation task is to define what each category means for the business.
Start with the feature that exists
Microsoft introduced memory in Foundry Agent Service in public preview in November 2025. The announcement describes extracting and retrieving user information and conversation summaries across interactions. That offers a managed foundation for continuity, with preview terms that teams need to evaluate before adopting it.
Keep the scope of that announcement clear. This design uses the documented memory concept and application controls; it does not assume that every retention, correction, or identity requirement is automatically solved by enabling memory.
The broader Agent Framework architecture provides orchestration and session building blocks. A workflow checkpoint, a conversation history, and a remembered customer preference serve different purposes. Treating them as one unstructured store makes retention and testing unnecessarily difficult.
The research argues for explicit evaluation
LongMemEval, introduced in 2024, uses 500 curated questions to examine long-term conversational memory. Its abstract reports an approximately 30% accuracy drop across sustained interactions in evaluated commercial assistants and long-context models. The benchmark tests extraction, cross-session reasoning, time, updates, and abstention. These findings describe its tested systems and conditions, not Foundry performance.
The older Lost in the Middle study also found that access to a long context did not ensure reliable use of relevant information, particularly when its position changed. A larger input window is therefore an option to evaluate, not proof that a memory problem has disappeared.
For an enterprise architect, the implication is practical. Test what the assistant recalls, when the remembered fact was true, and when it should decline to answer. Storage success alone is a weak acceptance criterion.
Separate business records from conversational memory
Imagine a hypothetical distributor using a support agent for repeat customers. The order-management system owns shipping addresses and account permissions. The memory layer might retain a preference for concise status summaries or a usual delivery-time discussion. It should not become an alternative system of record for the customer’s authorized receiving address.
For a new order, retrieve the current address from the authoritative system and ask for confirmation when the workflow requires it. The remembered preference can make the conversation smoother without bypassing the transaction’s normal validation.
This distinction also helps with data quality. If a customer says an address changed, the assistant can propose the appropriate account-update process. It should not silently write a conflicting address into memory and later present that as verified operational data.
Bind scope to authenticated identity
Derive memory scope from the application’s authenticated tenant and user identifiers. Never allow a natural-language request such as “use the profile for user123” to select a different person’s memory.
Decide how shared accounts work before launching. An account-level purchasing preference may be visible to a team, while a person’s accessibility preference should remain personal. Those are different scopes with different access rules.
Test isolation at retrieval and update boundaries. A successful test proves both that another user’s facts stay hidden and that the current user cannot overwrite them indirectly. Include tenant changes, reassigned accounts, and recycled application identifiers in the test plan.
Preserve origin and time
A useful memory item needs enough provenance to explain why it exists. My recommended record includes its source interaction, time recorded, applicable scope, fact type, and any validity period. Avoid retaining sensitive raw conversation content merely because it is convenient.
Represent corrections explicitly. If a customer first prefers phone calls and later requests email, retrieval should prioritize the current preference while respecting the chosen retention policy. A vague summary containing both statements leaves the model to guess which one governs.
Some memories should expire because the business meaning expires. A travel preference for a particular week is not a lifelong customer attribute. Where the chosen preview interface lacks a required lifecycle control, implement that control in the application or postpone that memory category.
Let users correct the assistant
Provide a practical way to inspect or challenge remembered information. “That is no longer correct” should trigger a defined correction path. The agent should acknowledge the change without promising deletion across every system unless the application has completed that operation.
Deletion needs its own verification. Check that removed information no longer appears through direct lookup, semantic retrieval, derived summaries, or copied application caches within the scope you control. Document any separately retained audit record and its purpose.
A support team also needs a way to investigate a disputed memory without unrestricted access to private conversations. Use narrowly scoped diagnostics and log metadata rather than exposing all remembered content by default.
Build a memory-specific test set
Create multi-session cases with known facts, deliberate corrections, similar customers, and questions with no answer in the history. Include temporal questions such as which instruction applied before a change.
Compare memory enabled and disabled on the same task set. Score correct recall, stale-fact use, unsupported recall, cross-user exposure, and appropriate abstention separately. An overall average can hide a serious isolation defect behind better preference recall.
Keep the retrieved memory items with the evaluation evidence when permitted. That lets engineers distinguish extraction failure from retrieval failure and reasoning failure. Changing the model when the wrong memory was retrieved may make the symptoms harder to reproduce.
Decide whether continuity is worth its cost
Measure avoided clarification turns and successful task completion alongside retrieval latency, storage, model calls, correction effort, and privacy operations. Use the business task as the denominator rather than the number of memories stored.
A sensible pilot begins with low-risk preferences in one authenticated application. Expand the scope after demonstrating useful recall, reliable correction, and effective separation between users.
The strongest memory design is selective. It remembers information with a clear purpose, retrieves it within the right identity boundary, and knows when a fresh business record or another question is more reliable.
References
Microsoft: Memory in Foundry Agent Service, November 25, 2025
Microsoft: Agent Framework architecture, February 19, 2026
