Building Long-Term Memory for AI Agents with Microsoft Foundry

A capable language model can still feel strangely forgetful. It may reason well inside one conversation, then act as if it has never met the user when a new session begins.

Microsoft Foundry Agent Service addresses that gap with Memory, a managed long-term memory capability for agents. Instead of replaying every previous message, the service extracts durable information, stores it outside the model, and retrieves relevant memories when a later interaction needs them.

That sounds simple, but reliable memory is not just a vector database. It is a lifecycle:

This article explains that lifecycle, how Foundry maps it to memory stores and scopes, how asynchronous updates affect application design, and what C# developers should account for while the capability remains in preview.

First, use the right mental model

An LLM does not remember previous sessions by itself. It responds to the context supplied with the current request. If an application appears to remember something from last month, another component retrieved that information and placed it in the model’s usable context.

Think of an agent as using two workspaces:

1. A desk: the current context window, containing instructions, recent messages, tool results, and other immediate state.
2. A filing system: persistent information stored outside the model and searched when relevant.

The desk is fast but finite. The filing system survives across sessions, but its contents must be selected, organized, retrieved, and governed.

This distinction prevents a common architectural mistake: treating a long transcript as long-term memory. A transcript records what happened. A memory system tries to preserve what remains useful.

CoALA: a useful vocabulary, not a product architecture

The Cognitive Architectures for Language Agents, or CoALA, paper proposes a conceptual framework for describing language agents. It separates working memory from long-term semantic, episodic, and procedural memory. It also describes action spaces and decision procedures, so it is broader than memory alone. Sumers et al., “Cognitive Architectures for Language Agents”

The taxonomy gives us a practical vocabulary:

Memory category	Useful interpretation	Example
Working memory	Information active during the current run	The latest user message and a tool result
Semantic memory	Facts and stable knowledge	“The user prefers C# examples”
Episodic memory	Summaries of prior events or interactions	“We previously diagnosed a token-expiration issue”
Procedural memory	Reusable instructions or ways of working	“Check the deployment region before proposing a model”

Foundry’s implementation does not reproduce the entire CoALA architecture. The comparison is still useful because Foundry currently exposes three long-term memory types that roughly align with the taxonomy:

• User profile memory for durable user facts and preferences
• Chat summary memory for distilled summaries of earlier conversations
• Procedural memory for reusable routines inferred from prior interactions

The mapping is approximate, not exact. CoALA is a research framework. Foundry Memory is a managed product capability with its own behavior and constraints.

It also helps to separate memory from knowledge grounding. Long-term memory captures information learned through interactions, typically about a user, prior conversation, or recurring procedure. Foundry IQ knowledge bases and file search serve a different role: grounding the agent in curated or user-provided content. Both may contribute to the final prompt, but they solve different problems.

What Foundry Memory actually does

A Foundry memory store is persistent storage plus a model-driven processing pipeline. When you create a store, you configure:

• a compatible chat model deployment, used to interpret conversation content and produce memories;
• an embedding model deployment, used to represent memories for semantic retrieval;
• enabled memory types and extraction options;
• optional defaults such as a time to live, or TTL, for newly created entries.

Microsoft recommends creating a dedicated memory store for each agent. That boundary helps keep extraction behavior, access patterns, and optimization concerns understandable. Microsoft Learn: Create and use memory

You can create and manage stores through the Foundry portal, the REST API, or supported SDKs. The current documentation provides usage paths for Python, C#, JavaScript or TypeScript, and Java. A store is also usable through the low-level APIs without attaching it to a Foundry prompt agent, which makes it a reusable service boundary for custom orchestrators.

Some store defaults are selected when the store is created. The preview documentation specifically warns that options such as procedural-memory enablement and default TTL may not support post-creation changes in every API version. Treat store configuration as version-sensitive and verify update support before depending on it operationally.

The memory pipeline

At a high level, Foundry performs three documented phases:

1. Extraction identifies useful information in the interaction.
2. Consolidation merges or resolves overlapping memories where applicable.
3. Retrieval searches for memories relevant to a later interaction.

Embeddings support semantic retrieval, but they are only part of the design. The chat model decides what is worth remembering and may consolidate related information. This is why memory is better understood as a model-driven knowledge pipeline than as a raw vector insert.

The source walkthrough also contrasts this pipeline with memory systems that perform entity linking into a graph. That can add explicit relationships between people, places, projects, and other entities. Foundry’s documented memory pipeline focuses on extracted memory items and semantic retrieval; it does not currently document a graph-based entity-linking stage for Memory. If your use case needs relationship traversal, treat that as a separate knowledge-graph capability rather than assuming the memory store provides it.

There is an important preview caveat: consolidation behavior can differ by memory type and may change. Do not assume that every memory type is deduplicated in exactly the same way. Test the behavior your application depends on.

Memory types and when to retrieve them

User profile memory

User profile memory holds comparatively stable facts or preferences, such as:

• preferred language;
• accessibility needs;
• default output format;
• recurring product preferences.

Retrieve profile information near the beginning of a conversation when it should shape the entire interaction.

You can guide extraction with the store’s user profile details instruction. A fitness assistant, for example, might retain workout preferences and dietary constraints while excluding unrelated personal details. This is not just an accuracy feature. It is a data-minimization control: the best way to protect unnecessary personal information is not to store it.

Chat summary memory

Chat summaries preserve the shape of earlier conversations without replaying the complete transcript. They are useful when the current message relates to a previous topic, decision, or unresolved thread.

Retrieve them in relation to the current discussion, typically per turn, rather than injecting every historical summary into every prompt.

Procedural memory

Procedural memory captures reusable ways of working. Examples include:

• “When preparing a migration plan, list prerequisites before the steps.”
• “For troubleshooting, ask for the correlation ID before recommending a retry.”

Retrieve procedural memories when the user asks for a recurring task or workflow. Avoid treating inferred procedures as unquestionable policy. They may be incomplete, outdated, or learned from an exceptional case.

Scopes are isolation boundaries

A scope partitions memories inside a store. Each scope has its own isolated collection of memory items. In a customer-facing agent, a stable customer identifier is a natural scope. In another design, the scope might represent an account or another durable application boundary.

A useful rule is:

When you attach the memory search tool to a prompt agent, Foundry can use {{$userId}} for per-user isolation. The service resolves the user from the x-memory-user-id header when supplied, or falls back to identity information in the Microsoft Entra token. When calling the low-level memory APIs directly, your application must provide the scope explicitly. Microsoft Learn: Understand scope

Because a scope is a plain string, your application can encode a composite boundary such as userId:applicationId when memories must not flow across applications. Keep the format stable and opaque, and avoid placing readable personal information in the key. Composite strings are an application convention, not a structured metadata model provided by the scope field.

A scope is an isolation key, not proof of authorization. Your application must still authenticate the caller, verify that the caller may access that identity, and prevent an arbitrary scope value from being substituted.

Current quota implications

As of August 7, 2026, the documented preview limits include:

• 100 scopes per memory store;
• 10,000 memories per scope;
• 1,000 search requests per minute;
• 1,000 update requests per minute.

The 100-scope limit is especially significant. It may rule out a simple “one store for every customer” design for larger applications. Because quotas can change during preview, check the current limits and region list before finalizing the topology.

Retrieval is prompt construction

The model cannot use a memory that remains in storage. Retrieval must bring relevant items into the agent’s active context.

A practical request path looks like this:

1. Receive the current user message.
2. Resolve the correct memory scope.
3. Search for relevant profile, summary, or procedural memories.
4. Add selected results to the agent context.
5. Call the model.
6. Submit the new interaction for memory processing.

This is often called context hydration. The phrase is useful because it highlights the real engineering task: building the smallest, most relevant context for the current decision. The complete context may combine recent chat history, grounded knowledge, tool output, and retrieved long-term memory.

When a memory store is attached to a prompt agent, Foundry exposes memory search as a tool. The model can decide whether to call that tool. With the low-level API, the application makes the decision explicitly by calling search itself. In either form, retrieval results still have to become model-visible context before they can influence the answer.

The low-level search operation is intentionally simple in the current preview. The application supplies a search query and scope and can limit the number of returned memories. The query is represented for semantic matching through the configured embedding model. Compared with systems that expose richer metadata or category filters, this leaves more retrieval policy in the application layer.

More retrieved memory is not automatically better. Irrelevant memories consume tokens and can distract the model. Incorrect memories can actively degrade answers. Treat retrieval quality as an evaluation problem, not as a storage checkbox.

Why updates are asynchronous

Low-level memory updates are long-running operations. An update request can move through states including:

• queued
• in_progress
• completed
• failed
• superseded

The API accepts the work and returns an update identifier that the application can poll. This keeps extraction and consolidation away from the user-facing response path. Depending on the submitted content, a completed update can report memory-item operations such as records being created, updated, or deleted. Microsoft Foundry API reference: Update memories

Do not design around the timing observed in a single demo. The source walkthrough saw operations complete in roughly five to twenty-three seconds and noted that documentation allowed longer processing. Those measurements are illustrative, not a service-level guarantee. Poll terminal status with a bounded timeout and handle delayed or failed work.

The resulting consistency model is best described as eventual:

• The current conversation already contains the latest statement in short-term context.
• The durable memory store may not reflect that statement immediately.
• A later conversation should see it after processing completes.

This separation improves responsiveness, but it changes how you test and reason about the system.

Testing asynchronous memory

A fragile integration test does this:

1. Submit a message containing a new preference.
2. Immediately start a new session.
3. Assert that the preference is recalled.

That test races the background pipeline. A reliable test should instead:

1. submit the memory update;
2. capture its update ID or polling location;
3. wait until the operation reaches completed;
4. start a fresh session;
5. verify recall from the same scope;
6. also verify that another scope cannot retrieve the memory.

Test both positive recall and isolation. An agent that remembers the right fact for the wrong user is worse than one that forgets.

C# integration patterns

The current Foundry documentation supports C# SDK operations for creating and managing stores, attaching memory to prompt agents, searching and updating memories, and managing individual memory items. The exact SDK surface is preview-sensitive, so pin package versions and follow the examples for the API version you deploy. Microsoft Learn: C# usage support and setup

At the application level, keep memory integration behind two explicit hooks:

Before the model call

• Resolve the authenticated user’s scope.
• Search for relevant memories.
• Filter or rank the results if necessary.
• inject selected memories into the agent context.

After the model call

• Submit the user and assistant interaction to the memory store.
• Record the returned update ID.
• Process completion or failure asynchronously.
• Avoid blocking the response unless the workflow explicitly requires a committed memory.

If you use Microsoft Agent Framework’s Foundry memory provider, the provider can encapsulate these pre-call retrieval and post-call update steps through the framework’s AI context-provider pattern. Conceptually, ProvideAIContextAsync runs before model invocation and searches for memories, while StoreAIContextAsync runs after the response and submits new context for asynchronous processing. The provider is a convenience wrapper over the Foundry project client and memory APIs, not a different memory engine.

The retrieved context is typically added with an instruction prefix that tells the model to consider the supplied memories. If your provider version allows that prefix to be configured, keep it explicit and neutral. Retrieved memories are context, not higher-priority system instructions.

Provider conveniences can also help tests and setup. The source example cleared all memories in a test scope before execution, created one conversation that stated “I live in Poland,” waited for pending updates to complete, then opened a new conversation and verified recall. Use the equivalent methods available in your pinned framework version rather than assuming preview method names remain stable.

If you use the Foundry SDK or REST API directly, implement the same lifecycle in your orchestration layer. Either way, keep the memory boundary visible in telemetry so you can distinguish retrieval failures, model failures, and update-pipeline failures.

Batching updates with update_delay

The low-level API supports update_delay, expressed in seconds. The current API reference documents a default of 300 seconds. Setting it to 0 requests immediate processing. If another update arrives during the delay, the waiting operation can be cancelled and the timer reset.

This is useful because extracting memory after every message is often wasteful. A user may refine or reverse a preference within the same conversation:

1. “Use a formal tone.”
2. “Actually, keep it conversational.”
3. “Conversational, but concise.”

Waiting briefly allows the memory pipeline to process a more complete interaction instead of turning every intermediate statement into durable state.

Choose the delay according to the experience:

• Interactive assistant: delay updates so the foreground response remains fast.
• End-of-session summarization: submit a compact, bounded conversation when the session closes.
• Explicit “remember this” command: use the documented synchronous remember-or-forget behavior when immediate memory management is required and supported by your integration.

Incremental updates and superseding work

The update API accepts previous_update_id. This links a request to earlier update work and enables incremental processing. If pending work is replaced, the earlier operation can reach superseded, with the response identifying the update that replaced it.

This matters during bursts of conversation. Instead of treating five quick messages as five unrelated extraction jobs, the application can preserve continuity between updates and avoid waiting for stale work.

Do not treat superseded as a generic failure. It is a terminal state indicating that newer work took precedence. Observability and tests should model it separately from failed.

Retention is not the same as “memory decay”

The original intuition behind memory decay is sensible: old information should not dominate forever. However, it is important to separate a general design goal from currently documented product behavior.

Foundry documents these concrete lifecycle controls:

• a store-level default TTL for newly created memories;
• item-level create, read, update, list, and delete operations;
• direct remember-or-forget commands;
• extraction and conflict resolution where applicable.

TTL is deterministic expiration. It is not semantic decay based on how often a memory is accessed. The current documentation does not establish a general, automatic “frequently used memories become stronger” mechanism, so applications should not rely on one.

The source demo observed that extracted memory text included wording such as “statement recorded on July 18, 2026.” Embedding temporal wording in the memory content can help a model interpret when a statement was true, even when structured timestamps also exist. That is a useful temporal-reasoning pattern, but the exact text generated by the preview service is implementation behavior, not a stable contract.

For time-sensitive facts, include time in the source interaction and test how retrieval handles updates. A statement such as “I am 40 years old” needs the date on which it was stated to remain interpretable later. For strict business state, such as an active subscription tier or a current shipping address, use the authoritative system of record instead of agent memory.

Memory introduces a new trust boundary

Long-term memory can improve continuity, but it can also preserve mistakes or malicious instructions. Microsoft specifically calls out prompt injection and memory corruption as risks.

At minimum:

• authenticate and authorize every memory scope;
• minimize what the extraction prompt is allowed to retain;
• avoid storing credentials, secrets, or unnecessary sensitive data;
• support deletion and expiration;
• log memory writes and retrievals without leaking protected content;
• adversarially test whether untrusted content can create harmful procedural memories;
• distinguish remembered preference from authoritative enterprise data.

A useful principle is:

Also note a current platform limitation: VNet integration is not supported for memory stores in the latest preview. That can be a decisive constraint for regulated or network-isolated environments.

Foundry Memory or Mem0?

Foundry Memory is not the only way to add persistent context. Mem0 is a separate memory platform with conversation, session, user, and organizational layers, plus identifiers and metadata for retrieval. Mem0 documentation: Memory types

The comparison is architectural rather than absolute:

Consideration	Microsoft Foundry Memory	Mem0
Platform fit	Native to the Microsoft Foundry and Azure ecosystem	Designed as a framework-independent memory layer
Main isolation model	Memory store plus a required string scope; applications may use a composite key	User, run/session, application or organization layers, plus metadata patterns depending on product and API
Managed extraction	Built into the Foundry memory pipeline	Built into Mem0’s hosted platform and available through its open-source ecosystem
Current maturity signal	Public preview in Foundry Agent Service	Independent product with hosted and open-source options
Best fit	Teams prioritizing Foundry integration and managed Azure tooling	Teams prioritizing portability or richer cross-framework memory partitioning

Do not choose from a feature checklist alone. Prototype the same workload against both options and evaluate:

• recall precision;
• incorrect-memory rate;
• update latency;
• isolation behavior;
• deletion semantics;
• observability;
• regional and networking requirements;
• operational ownership.

Practical design rules

If you remember only eight things, remember these:

1. Keep short-term state and long-term memory separate. A context window is not durable memory.
2. Treat memory as a pipeline, not a database write. Extraction and consolidation are model-driven operations.
3. Use one clear owner per scope. Resolve that scope from authenticated identity where possible.
4. Retrieve selectively. More memory can make an answer worse.
5. Design for eventual consistency. Do not block normal chat responses on background consolidation.
6. Poll in tests. Never assume an accepted update is already searchable.
7. Use TTL and deletion intentionally. Retention should reflect the nature of the information.
8. Keep authoritative state elsewhere. Agent memory is contextual knowledge, not a transactional source of truth.

Final perspective

Long-term memory changes an agent from a stateless answer generator into a system that can build continuity over time. The hard part is not remembering everything. The hard part is remembering the right information, for the right owner, for the right duration, and retrieving it only when it helps.

Microsoft Foundry provides a strong managed foundation: scoped stores, multiple memory types, model-driven extraction, semantic retrieval, asynchronous updates, TTL, and item-level lifecycle operations. It also remains a preview capability with important limits, networking constraints, and behavior that can evolve.

The right approach is therefore deliberate rather than magical. Start with a narrow memory contract, make the asynchronous lifecycle explicit, test isolation and stale-memory scenarios, and expand only when measured retrieval quality justifies it.

References

• Memory in Microsoft Foundry Agent Service
• Create and use memory in Foundry Agent Service
• Update memories API reference
• Cognitive Architectures for Language Agents
• Mem0 memory types