Building Unified AI Agents with Microsoft Foundry IQ and Python

A large language model can produce fluent answers without knowing anything about your private documents, current policies, or operational data. That gap is why enterprise AI is rarely just a model-selection problem. It is a context-engineering problem.

The difficult part is not merely finding a document. An agent may need to interpret a conversation, split a complex question into several searches, choose appropriate sources, combine keyword and vector results, preserve the caller’s permissions, and return evidence that another model can use safely.

Microsoft Foundry IQ is Microsoft’s managed knowledge layer for that job. It uses Azure AI Search as its retrieval foundation and packages knowledge sources, retrieval policy, permission enforcement, and grounded output into reusable knowledge bases. Multiple agents can call the same knowledge base instead of each agent implementing its own retrieval stack.

This article explains the architecture, the ingestion and ranking pipeline, the retrieval-effort settings, MCP integration, and the main ways a Python agent can consume Foundry IQ. It also separates generally available building blocks from capabilities that remain in preview as of July 29, 2026.

Microsoft IQ: four kinds of context

Microsoft describes Microsoft IQ as a shared intelligence layer for Copilots and agents. Its four capabilities address different kinds of context:

This does not mean every agent needs all four. A policy assistant may need only Foundry IQ. A sales-planning agent might combine Work IQ, Fabric IQ, and selected web context. The useful mental model is not “four products to attach,” but four context planes with different semantics, freshness, and authorization boundaries.

This article stays centered on Foundry IQ: the plane for turning documents and other knowledge sources into a retrieval surface that agents can share.

Reproduce the walkthrough

The session behind this article is accompanied by Microsoft’s public microsoft/iqdeepdive repository. It includes Jupyter notebooks, Python agents, infrastructure definitions, and examples for Foundry IQ, Work IQ, and Fabric IQ. The Foundry IQ materials demonstrate direct knowledge-base retrieval, a custom agent tool, the knowledge base’s MCP endpoint, Foundry Toolbox, and hosted-agent deployment.

Running the examples requires an Azure subscription and the services, model deployments, roles, and—where applicable—product access used by each notebook. Treat checked-in notebook output as a way to study the flow when you cannot provision every dependency yourself; do not assume that access to the repository grants access to preview services or licensed IQ capabilities.

Foundry IQ is more than a vector index

Calling Foundry IQ “vector search for agents” understates the architecture.

A vector index is a storage and similarity-retrieval mechanism. A Foundry IQ knowledge base is an orchestration boundary. It defines which knowledge sources are available and how retrieval should behave. At query time, agentic retrieval can plan searches, select sources, execute subqueries, merge and rerank matches, and return either supporting passages or a synthesized answer with citations.

Think of the knowledge base as a research desk:

• The model or calling agent provides the question and, where relevant, conversation history.
• A planner turns the request into focused research tasks.
• Each knowledge source behaves like a specialist collection.
• Retrieval runs searches across the selected collections.
• Ranking reconciles the candidates into one evidence set.
• The response gives the agent grounded material and source attribution.

The pipeline can be summarized as follows:

1. Understand the request. Low and medium reasoning effort can use an LLM to interpret the request and create targeted subqueries.
2. Select knowledge sources. The retrieval layer chooses sources that match the intended topic.
3. Search in parallel. Indexed sources can use keyword, vector, or hybrid retrieval; remote sources may be queried through their own interfaces.
4. Merge and rerank. Candidate lists are combined and the most relevant evidence is promoted.
5. Return grounded output. The caller can receive extractive results with citations or use answer synthesis where configured.
6. Iterate when necessary. With medium reasoning effort, the system can perform another retrieval round when the initial evidence is insufficient.

The important architectural shift is reuse. Instead of wiring every Python agent directly to every index, storage account, SharePoint location, or remote service, you can expose a curated knowledge base as a stable retrieval contract.

How documents become agent-ready knowledge

For indexed sources, Foundry IQ and Azure AI Search automate much of the ingestion work: content extraction, chunking, embedding generation, metadata creation, indexing, and recurring refresh for supported sources.

1. Extraction

The pipeline first converts source files into content that retrieval can use. Extraction quality matters because search cannot recover information that never entered the index. Tables, scanned pages, images, layout, and unusual file encodings deserve explicit testing; “the file uploaded successfully” is not evidence that every important element was extracted correctly.

2. Chunking

Large documents must be divided into retrievable units. There is no universally correct chunk size.

A 500-to-1,000-token range can be a reasonable experiment, but it should not be treated as a Foundry IQ rule. Small chunks can improve precision while losing surrounding context. Large chunks preserve context but may dilute relevance and consume more model input. Document structure, headings, table boundaries, overlap, and the questions users actually ask should drive tuning.

3. Embeddings and searchable text

Embeddings represent semantic similarity, while searchable text supports exact and lexical matching. Keeping both is essential because users ask both conceptual and literal questions.

• “How do we handle a damaged shipment?” benefits from semantic similarity.
• “What is policy code RTN-0427?” depends on exact matching.

4. Indexing and metadata

A useful index commonly retains chunk content, a stable chunk identifier, a parent-document identifier, source location, and metadata needed for filtering, citation, freshness, and access control. The exact schema depends on the knowledge source and retrieval requirements; it is not limited to one fixed set of fields.

Metadata is operationally important. Without a reliable document identity and source URI, an agent may produce an answer that sounds grounded but cannot help the reader inspect the evidence.

File knowledge sources and indexed storage sources

The original choice is not simply “prototype versus production.” It is better understood as managed upload versus managed synchronization from an external store.

A file knowledge source can be excellent for a Python proof of concept because it avoids provisioning a separate storage container and indexer pipeline. It is not “zero configuration,” however: the application still needs an Azure AI Search service, authentication, a knowledge source, a knowledge base, compatible API versions, and appropriate model and role configuration.

For content that already lives in Blob Storage or ADLS Gen2—or when scale, lifecycle management, and recurring refresh matter—the storage-backed knowledge source is the more natural design.

The indexer and skillset pipeline

Blob and ADLS Gen2 are common document sources, but Azure AI Search indexers can also pull from supported structured and semi-structured sources such as Azure SQL Database, Azure Cosmos DB, Azure Table Storage, and OneLake. An indexer has one source and one target index; multiple indexers can populate the same index when the schema supports that design.

A typical enrichment pipeline contains four resource types:

data source -> indexer -> skillset -> search index

• The data source defines the external connection.
• The indexer schedules and coordinates extraction, field mapping, change detection, and deletion handling where the connector supports them.
• The skillset transforms content while it is in the enrichment pipeline.
• The search index stores the searchable text, vectors, identifiers, and metadata used at query time.

Indexers can run on demand or on a schedule. Many supported sources provide change detection, so a later run can process additions or updates rather than rebuilding everything. Deletion detection depends on the connector and its configuration; it should be tested rather than assumed.

Built-in skills cover common operations. The Text Split skill can divide content into pages or chunks, and Azure OpenAI embedding skills can generate vectors. The preview Azure Content Understanding skill can extract text and images—including location metadata that preserves where an image appears relative to surrounding content—and can also perform chunking. This is useful when figures, tables, and layout carry meaning that plain-text extraction would lose.

A skillset can also call a custom skill hosted behind an HTTP endpoint, commonly implemented with Azure Functions. Use a custom skill only when built-in processing cannot satisfy the requirement—for example, when you need a specialized parser, enrichment algorithm, or embedding service. Custom code increases operational responsibility, latency, and failure modes, so start with built-in skills and extend deliberately.

Why hybrid retrieval beats a vector-only mental model

Vector search is powerful, but semantic similarity is not the same as correctness. Exact identifiers, dates, names, error codes, and product SKUs often favor lexical search. Azure AI Search therefore supports hybrid search, where full-text and vector queries run in parallel.

The stages are:

1. Full-text retrieval identifies lexical matches, using text-ranking techniques such as BM25.
2. Vector retrieval identifies semantically similar chunks by comparing embeddings.
3. Reciprocal Rank Fusion (RRF) combines independently ranked result lists into one list.
4. Semantic ranking, when enabled, applies a secondary language-model-based ranking stage to the initial candidates.

RRF does not attempt to compare incompatible raw scores from keyword and vector systems. It rewards documents that rank highly across one or more input lists. Conceptually, each appearance contributes a value related to:

1 / (rank + k)

The contributions are summed, and the combined score establishes the fused order.

Two corrections are important:

• Semantic ranker is not accurately described as a guarantee that “the top 10 results are truly relevant.” Ranking models improve relevance statistically; they do not certify truth.
• The semantic ranker is a Microsoft language-understanding reranker over a pre-ranked candidate set. Calling it a generic “cross-encoder” may be directionally intuitive, but that label is less precise than the product documentation.

The resulting mental model is simple: keyword retrieval protects precision, vector retrieval expands semantic recall, RRF combines their evidence, and semantic ranking improves the final ordering.

Retrieval reasoning effort: who owns the search plan?

Foundry IQ knowledge bases support minimal, low, and medium retrieval reasoning effort. These settings define how much planning the retrieval layer performs.

Minimal effort does not mean “bad retrieval.” It means the caller owns more of the reasoning. It can merge results from multiple explicit queries without asking another LLM to rewrite them.

Low effort gives the knowledge base responsibility for query planning and source selection. Medium effort adds iterative retrieval when the first pass does not meet relevance thresholds.

More reasoning is not automatically better. Medium effort can improve difficult answers, but it can add latency, model use, and more opportunities for an incorrect query reformulation. Evaluate it with representative questions rather than selecting it by intuition.

The session also discussed a possible future auto mode, but that was presenter speculation—not a documented product commitment. Today, architecture should be based on the documented minimal, low, and medium settings.

Querying a knowledge base: answers, references, and activity

Creating a knowledge base is only half the job. At runtime, an application calls its retrieve action—or the equivalent MCP tool—with a user request, conversation, or explicit search intents. The response can contain three distinct kinds of information:

1. Retrieved evidence and references. References identify the source material used by retrieval and provide the metadata needed to build inspectable or clickable citations.
2. A synthesized answer. When answer synthesis is configured, the knowledge base’s LLM can compose a grounded response from the retrieved evidence.
3. Retrieval activity. The activity details reveal planning and execution steps such as generated subqueries, selected sources, result merging, and synthesis.

These outputs serve different consumers. The end user usually needs the answer and readable citations. The agent may need the evidence and references so it can reason or decide whether to call another tool. Engineers and evaluators need retrieval activity to diagnose why the pipeline succeeded or failed.

For example, a file knowledge source can be created from an uploaded Markdown document, attached to a knowledge base, and queried with a conversational request. Upload is not instantaneous because the service must extract, chunk, embed, and index the content. The original demonstration completed a small upload in tens of seconds and a query in several seconds, but those timings are observations from one demo—not service guarantees.

A multi-source knowledge base works the same way, except the activity can show several generated queries dispatched across multiple index knowledge sources. Parallel execution reduces the latency that would result from querying sources sequentially, although total time still depends on planning, the slowest selected source, reranking, and optional synthesis.

Retrieval instructions become important at scale

Source selection cannot rely on names alone. When a knowledge base contains several domains, give it clear retrieval instructions describing what each source contains and when it should be used. For example:

Use the HR policy source for company benefits and enrollment rules.
Use the approved public-web source only for external industry context.
Do not use public web results as authority for internal company policy.

Instructions improve routing, but they are not access controls. Authorization must still be enforced by the underlying identity, connection, source permissions, and query-time security model.

Extending a knowledge base with MCP

Model Context Protocol (MCP) gives agents a standard way to discover and call tools. Foundry IQ participates in MCP in two directions:

1. A Foundry IQ knowledge base can expose retrieval through an MCP endpoint, allowing an agent to call the knowledge base as a standardized tool.
2. An MCP server can be configured as a remote knowledge source, allowing agentic retrieval to call an external system at query time.

The second pattern is how a remote service—including an eligible web intelligence service—can participate in a Foundry IQ retrieval plan. The knowledge base can combine indexed internal content with remote context instead of requiring the agent to coordinate each source separately.

However, this boundary needs careful treatment:

• MCP-server knowledge sources are preview in the 2026-05-01-preview API.
• Remote calls can move data across service, compliance, or geographic boundaries.
• Authentication, tool descriptions, timeouts, error handling, and trust in the remote server become part of retrieval quality.
• An external MCP server can fail, return malicious content, or trigger cascading tool behavior. It should be vetted and monitored like any other external dependency.

Where Web IQ fits

Web IQ is Microsoft’s context plane for fresh web information. Architecturally, it complements Foundry IQ: internal, curated knowledge answers “what does our organization know?”, while web context answers “what has changed outside it?”

Do not assume that every tenant, region, SDK, or API version exposes Web IQ through the same integration path. The safe implementation approach is to verify current Web IQ access and then use a documented Foundry or MCP connection supported by that environment. Treat web retrieval as an external, less-trusted source and preserve URLs and freshness metadata so downstream answers remain inspectable.

In the source demonstration, Web IQ was presented as a private-preview MCP service with web-oriented tools that could return fields such as title, URL, content, crawl date, and language in agent-friendly formats. That status and response shape are time-sensitive. Verify current access terms and tool schemas before implementation; do not copy a preview key or header pattern from a demo into a production design.

A remote MCP knowledge source behaves differently from an indexed source. Its content is retrieved at runtime, incorporated into the candidate set, and can participate in final reranking, but it is not automatically copied into your Azure AI Search index. That preserves freshness and avoids duplicating the remote corpus, while making the external service’s availability and latency part of the request path.

Three ways to connect a Python agent

An agent is not merely “an LLM running tools in a loop,” but that is a useful starting abstraction. In practice, the runtime also needs state, identity, authorization, failure handling, tracing, and an explicit stopping condition.

Your Python integration can use one of three patterns.

Pattern 1: Call the knowledge-base API directly

Your code invokes Azure AI Search’s knowledge-base retrieval API, supplies the request, and handles the response. This gives you the most control over retries, payloads, filters, logging, and how citations enter the agent’s context.

This pattern fits applications that already have an orchestration layer or need deterministic control over the retrieval boundary. Minimal effort is often a sensible starting point when the agent itself owns query decomposition, but low or medium effort can still be selected deliberately.

A framework-neutral Python shape looks like this:

from dataclasses import dataclass
from typing import Any

@dataclass
class Evidence:
    text: str
    source_url: str | None
    metadata: dict[str, Any]

async def retrieve_evidence(question: str, history: list[dict]) -> list[Evidence]:
    """Call the Foundry IQ knowledge-base retrieve endpoint.

    Implement authentication with Microsoft Entra ID, send the request using
    the API version selected for your environment, and map returned references
    into your application's evidence model.
    """
    raise NotImplementedError

The abstraction is intentional. SDK names, request shapes, and preview API surfaces can change. Keep Foundry-specific transport code behind a small adapter so the rest of the agent depends on a stable internal Evidence model.

The session’s custom-tool pattern accepted a list of queries and returned the retrieval response, references, and activity details. That is especially useful with minimal effort: the outer agent writes the queries and can inspect the retrieval trace before deciding whether another tool call is needed. It also lets the agent combine knowledge retrieval with local tools—for example, a current-date function or an enrollment-deadline calculator—without forcing those deterministic operations into the knowledge base.

Query generation is itself an evaluation target. Different models can turn the same request into very different searches, so capture generated queries and test whether they retrieve the expected evidence. Tool instructions can require decomposition into a bounded number of focused queries when a single broad query performs poorly.

Pattern 2: Attach the knowledge base as an MCP tool

A Foundry IQ knowledge base exposes an MCP retrieval surface. A compatible agent can discover and invoke that tool, using Microsoft Entra authentication or the connection mechanism supported by the host.

This reduces custom tool-schema code and makes the integration more portable across MCP-capable runtimes. It also shifts some control from your Python function to the agent’s tool-selection behavior, so tool descriptions and agent instructions must be tested carefully.

The knowledge-base server exposes a retrieval tool commonly surfaced as knowledge_base_retrieve. The client must acquire the appropriate Microsoft Entra token and send it to the endpoint; the endpoint name alone does not bypass authorization. In production, prefer managed identity or another tightly scoped credential flow over embedded keys or developer credentials.

Pattern 3: Use Foundry Toolbox

Foundry Toolbox can present tools to a hosted agent through one managed tool connection. A toolbox may expose a Foundry IQ knowledge base and other capabilities that the agent is authorized to use. Microsoft’s Foundry IQ hosted-agent quickstart, for example, connects a toolbox to the knowledge base’s MCP endpoint and lets the hosted agent call the resulting retrieval tool with its managed identity.

Toolbox is useful when the agent needs several managed capabilities and you want centralized connection and identity handling. It is not a reason to give every agent every tool. A smaller, task-specific tool surface is easier to secure, evaluate, and debug.

The example toolbox in the session combined three capabilities:

• the Foundry IQ knowledge base through its MCP endpoint;
• a web-search tool that is distinct from Web IQ and should not be assumed to return identical results; and
• Code Interpreter for sandboxed Python execution.

The toolbox is itself consumed through one MCP-compatible endpoint. This makes a curated tool collection reusable across agents and lets its owner change or version the collection without hard-coding every downstream connection into each agent. Authentication is centralized at the toolbox connections, but authorization design still matters: the toolbox should expose only the tools and credentials appropriate for its consumers.

From a notebook to Foundry Agent Service

Retrieval code running in a notebook proves the search path, not the production system. A deployed agent needs a stable endpoint, identity, scaling, session behavior, observability, and lifecycle management.

Foundry Agent Service supports both prompt agents and hosted agents. Prompt agents are service-defined agents centered on model instructions and configured tools. Hosted agents are the bring-your-own-code path for cases where orchestration, framework choice, custom protocols, compute settings, or state behavior must live in your application.

Hosted agents in Foundry Agent Service provide a managed runtime for custom agent code. You can package an agent as a container, or use supported source-code deployment paths where available. The service assigns a dedicated Microsoft Entra agent identity, exposes an endpoint, and manages scaling, session persistence, observability, and versions. Hosted code runs in an isolated sandboxed environment; avoid treating that isolation as permission to execute arbitrary untrusted code without additional safeguards.

For Microsoft Agent Framework applications, the Foundry hosting integration can expose either:

• the OpenAI-compatible Responses protocol at /responses, which is the natural starting point for conversational agents; or
• the more general Invocations protocol at /invocations, for custom JSON payloads, webhook-style processing, or non-conversational workloads.

This is more precise than saying the Responses API is merely a “compatibility layer.” In the hosted-agent design, it is a supported invocation protocol with conversation history, streaming, and session behavior managed by the platform.

Microsoft Foundry Hosted Agents are generally available, although individual language integrations and deployment conveniences can still be prerelease or preview. Check the status of the specific Python hosting package and deployment path you use.

Observability: debug the retrieval chain, not only the final answer

When a grounded answer is wrong, the generation model may not be the root cause. The failure may have happened much earlier:

• The indexed document was stale.
• A table or scanned page was extracted incorrectly.
• Chunk boundaries separated the question from the answer.
• Query planning produced weak subqueries.
• Source selection skipped the authoritative collection.
• Permission enforcement correctly removed a document the developer expected to see.
• The right passage was retrieved but ranked too low.
• The final model ignored or misread good evidence.

Trace the system as a chain:

user request
  -> agent decision
  -> generated retrieval query or subqueries
  -> selected knowledge sources
  -> retrieved and reranked evidence
  -> citations and metadata
  -> final model response

Foundry provides tracing and observability capabilities for agents, and hosted agents manage platform-level instrumentation and logs. Your application should also record the retrieval request, selected source identifiers, latency, result count, citations, and relevant correlation IDs—without logging sensitive content indiscriminately.

Rather than relying on one vaguely defined “ultimate activity log,” design explicit telemetry for each stage. That makes it possible to distinguish retrieval failure from generation failure and to evaluate the system with measurable signals such as citation correctness, retrieval recall, answer groundedness, latency, and tool-error rate.

The source session correctly emphasized the activity log as a valuable debugging aid. It can show whether planning produced poor queries, whether an unexpected source was selected, and how synthesis followed retrieval. It cannot, by itself, prove that ingestion was correct or that an answer is factually complete. Pair it with index inspection, reference validation, traces, and a repeatable evaluation dataset.

Hosted-agent traces use OpenTelemetry-compatible instrumentation, making tool calls and retrieval operations visible in the Foundry experience. Evaluations can then be attached to representative runs, but managed evaluation features do not remove the need to define expected behavior, authoritative answers, failure thresholds, and regression gates.

Publishing to Microsoft Teams and Microsoft 365 Copilot

A tested Foundry agent version can be published to Microsoft Teams and Microsoft 365 Copilot from the Foundry portal. Publishing uses the agent’s stable endpoint and requires permissions to create and configure the supporting Azure Bot Service resource. Foundry project roles alone do not grant those Azure Bot Service permissions.

The portal experience is often described as one-click publishing, but the operational reality still includes prerequisites:

• an active, tested agent version;
• the appropriate Foundry role;
• Azure permissions for the bot resource and channels;
• tenant-level approval and distribution decisions where required; and
• a review of the additional Microsoft 365 and Teams data flows.

For projects with public network access disabled, the portal publishing button is not available; Microsoft documents a REST-based publishing path with additional networking configuration.

Publishing is therefore a distribution step, not a substitute for identity design, testing, or production readiness.

A practical architecture for Python agents

A robust design separates responsibilities:

Client or channel
    |
    v
Hosted Python agent
    |-- orchestration, state, and tool policy
    |-- Foundry IQ retrieval tool/API
    |      |-- indexed knowledge sources
    |      |-- remote MCP knowledge sources (preview)
    |      `-- permission-aware grounded output
    |-- optional task-specific tools
    `-- traces, evaluations, logs, and citations

Use this implementation sequence:

1. Define the questions and authoritative sources. Retrieval quality cannot exceed source quality.
2. Choose the knowledge-source type. Use direct file upload for bounded preview scenarios; use source-backed ingestion when content already has a durable repository or requires recurring synchronization.
3. Inspect extraction and chunking. Test tables, images, headings, metadata, and citations—not just upload success.
4. Establish a hybrid-search baseline. Include exact identifiers and semantic questions in the evaluation set.
5. Select retrieval effort deliberately. Start with minimal when the agent owns planning; use low when Foundry IQ should plan; test medium for genuinely multi-hop questions.
6. Choose the integration contract. Direct API for maximum control, MCP for standard tool interoperability, or Toolbox for managed multi-tool composition.
7. Use Microsoft Entra ID and least privilege. Preserve user or agent identity through the retrieval path where end-to-end permission enforcement is required.
8. Evaluate the chain. Measure retrieval, citations, groundedness, latency, and failure behavior separately.
9. Deploy and publish only after the retrieval contract is stable. A polished channel cannot compensate for unreliable evidence.

Final perspective

The most important Foundry IQ idea is not embeddings, MCP, or even agentic retrieval in isolation. It is the knowledge base as a reusable enterprise boundary.

That boundary lets platform teams curate sources and retrieval behavior once, while Python agents consume a consistent, permission-aware evidence service. Hybrid retrieval protects both exactness and semantic recall. Retrieval reasoning effort determines whether planning lives in the agent or the knowledge layer. MCP standardizes tool access. Foundry Agent Service supplies the managed runtime needed to operate custom agents beyond a notebook.

The result is not an omniscient agent. It is something more useful: an agent with an explicit research process, inspectable evidence, controlled access, and a deployment path that engineers can evaluate and improve.

References

• Microsoft IQ documentation
• Microsoft IQ Deep Dive sample repository
• What is Foundry IQ?
• Foundry IQ FAQ
• Create a knowledge base in Azure AI Search
• Create a file knowledge source
• Create a Blob knowledge source
• Azure AI Search indexer overview
• Azure AI Search skillset concepts
• Azure Content Understanding skill
• Create an MCP Server knowledge source
• Hybrid search overview
• Reciprocal Rank Fusion in Azure AI Search
• Semantic ranking overview
• Connect Foundry agents to Foundry IQ
• Foundry IQ hosted-agent quickstart with Toolbox
• Create, test, and deploy a Foundry Toolbox
• Code Interpreter for Foundry agents
• Hosted agents in Foundry Agent Service
• Host Microsoft Agent Framework agents in Foundry
• Publish agents to Microsoft 365 Copilot and Teams