Foundry IQ, the knowledge layer that stops every agent from rebuilding its own RAG

Every team that builds more than one AI agent eventually hits the same problem. The first agent gets its own retrieval pipeline with data connectors, chunking logic, embeddings, routing, and permission enforcement. The second agent needs to answer from the same documents, so the pipeline gets duplicated. By the third agent, there's a tangle of fragmented, siloed pipelines that are expensive to maintain and inconsistent in quality, with each one rebuilding the same foundation from scratch.

Foundry IQ is Microsoft's answer to that problem. Rather than adding more tools to the pipeline, it moves the retrieval layer out of each agent and into a shared, managed knowledge layer that multiple agents can query through a single API. The agents stop owning retrieval; they delegate it.

Before going further, it's worth clarifying a naming detail that causes confusion. Microsoft ships three distinct IQ products under the Microsoft Foundry umbrella. Foundry IQ handles enterprise documents and knowledge sources, which is what this article is about. Work IQ is a contextual intelligence layer for Microsoft 365 that captures signals from documents, meetings, chats, and workflows. Fabric IQ models business data in OneLake and Power BI so agents can reason over analytics. They're separate products with different semantics, freshness characteristics, and authorization boundaries. A policy assistant may need only Foundry IQ, while a sales-planning agent might combine Work IQ, Fabric IQ, and web context. This article stays focused on Foundry IQ.

For the official overview of what Foundry IQ is and how it fits into Microsoft Foundry, the Microsoft Learn documentation is the authoritative source: What is Foundry IQ?

What Foundry IQ is built on

Foundry IQ is built directly on Azure AI Search. This is the part most introductions skip over, and it matters for architecture decisions. A Foundry IQ knowledge base is not a separate service sitting next to Azure AI Search. It's a top-level object that lives on your Azure AI Search service and orchestrates agentic retrieval. When you create a knowledge base, you're creating an object in Azure AI Search that defines which knowledge sources to query and how retrieval behaves at query time.

The practical consequence is that you don't choose between Foundry IQ and Azure AI Search. Foundry IQ is the new interface for building agents on top of Azure AI Search, wrapping the underlying search capabilities in a managed, permission-aware, multi-agent-ready layer.

To explore the agentic retrieval capabilities that underpin knowledge bases, the Azure AI Search documentation covers the model in detail: Create a knowledge base in Azure AI Search

Knowledge sources and knowledge bases

Two concepts are foundational to how Foundry IQ is structured. A knowledge source is a connection to data, whether that's an Azure Blob Storage container, a SharePoint site, an OneLake lakehouse, an existing Azure AI Search index, or a web source accessed through the public internet or an MCP server. A knowledge base bundles one or more knowledge sources, up to ten per knowledge base, plus the parameters that control retrieval behavior. The knowledge base is what agents actually query.

The distinction matters because multiple agents can share the same knowledge base. An HR assistant and a policy assistant might both need access to the same document library. With a traditional RAG approach, each agent builds its own pipeline to that library. With Foundry IQ, both agents point at the same knowledge base and the retrieval layer handles routing, permissions, and answer synthesis once.

Knowledge sources come in two classes. Indexed sources are ingested into a search index on your Azure AI Search service, where the content is chunked, vectorized, and kept fresh through an indexer schedule. This class includes existing Azure AI Search indexes, Azure Blob Storage containers, and OneLake lakehouses, and these are generally available in the 2026-04-01 REST API. Federated sources are queried at runtime without pre-ingestion, including web sources and MCP servers, and these remain in preview in the 2026-05-01-preview and 2026-08-01-preview APIs.

GA versus preview: what this means for production

Some of the most useful Foundry IQ features are still in preview, and the distinction matters when deciding what to build on. The 2026-04-01 REST API provides a production-endorsed path with general availability for knowledge bases and core indexed knowledge sources. Features that remain in preview as of the time of writing include answer synthesis using an LLM for query planning and response generation, configurable reasoning effort tiers, document-level permissions, multi-turn conversational retrieval, and most of the portal-based management experience. The Azure portal and Microsoft Foundry portal continue to provide preview-only access to all agentic retrieval features regardless of which API version you use.

For a proof-of-concept or a hackathon project, the preview features are usable and the free tier of Azure AI Search is sufficient to get started. For a production deployment with SLA requirements, it's worth reviewing which specific capabilities you need against the GA boundary before committing to an architecture.

The product page for Foundry IQ on Azure has a current overview of generally available capabilities: Foundry IQ on Azure

Creating a knowledge base programmatically

The Python SDK for working with Foundry IQ knowledge bases uses azure-search-documents version 2026-05-01-preview or later for the knowledge base surface, combined with azure-ai-projects version 2.0.0 or later for the agent integration. Both are available on PyPI with no additional package feeds.

from azure.identity import DefaultAzureCredential
from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
    KnowledgeBase,
    KnowledgeSource,
    SearchIndexKnowledgeSource,
)

credential = DefaultAzureCredential()
search_endpoint = "https://your-search-service.search.windows.net"

index_client = SearchIndexClient(
    endpoint=search_endpoint,
    credential=credential,
    api_version="2026-05-01-preview",
)

knowledge_source = KnowledgeSource(
    name="company-docs-source",
    type=SearchIndexKnowledgeSource(
        index_name="company-documents",
    ),
)

knowledge_base = KnowledgeBase(
    name="company-docs-kb",
    description="Internal company documentation for agent grounding",
    knowledge_sources=[knowledge_source],
)

result = index_client.create_or_update_knowledge_base(knowledge_base)
print(f"Knowledge base created: {result.name}")

This creates a knowledge base that wraps an existing Azure AI Search index. The knowledge base inherits the permissions model of the underlying search service, so access control lists configured on the index are enforced at query time without additional configuration.

Connecting an agent to Foundry IQ

Once the knowledge base exists, connecting a Foundry Agent to it requires declaring the knowledge base as an MCP tool in the agent configuration. The Foundry IQ MCP endpoint is exposed per knowledge base and any MCP-compatible agent can use it.

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    Agent,
    McpTool,
    AgentMcpToolDefinition,
)

project_client = AIProjectClient(
    endpoint="https://your-foundry-project.services.ai.azure.com/api/projects/your-project",
    credential=DefaultAzureCredential(),
)

mcp_tool = McpTool(
    connection_id="your-search-service-connection-id",
    tool_names=["knowledge_base_retrieve"],
    knowledge_base_name="company-docs-kb",
)

agent = project_client.agents.create_agent(
    model="gpt-4.1-mini",
    name="document-assistant",
    instructions="""
        You are an assistant that answers questions about company policy.
        When answering, always use the knowledge base to retrieve relevant information.
        You must never answer from your own training knowledge when company-specific
        information is available in the knowledge base.
        Always cite the source documents in your responses.
    """,
    tools=[AgentMcpToolDefinition(mcp=mcp_tool)],
)

print(f"Agent created: {agent.id}")

The knowledge_base_retrieve tool is what allows the agent to query the knowledge base through agentic retrieval. When the agent receives a question, it uses this tool to plan queries, run parallel searches across the configured knowledge sources, and aggregate results with citations before generating a response.

How agentic retrieval works

The retrieval process that happens when an agent queries a knowledge base is more than a keyword or vector search. The agentic retrieval engine analyzes the incoming query, decomposes it into sub-queries when necessary, searches the configured knowledge sources in parallel using a combination of keyword, vector, and hybrid search depending on the source type, enforces user permissions at query time by checking access control lists and Microsoft Purview sensitivity labels, and returns extractive content with citations that trace each answer fragment back to its source document.

The reasoning effort applied to this process is configurable. Lower effort tiers use fewer tokens and respond faster at the cost of retrieval quality for complex multi-part questions. Higher effort tiers apply more query planning and source selection logic but consume more tokens per request. The appropriate tier depends on the agent's use case and the complexity of the questions it needs to answer.

For production agents, the permission enforcement behavior is particularly relevant. Foundry IQ synchronizes access control lists for supported knowledge sources and enforces them at query time, which means the agent never returns content that the querying user doesn't have permission to see, even if that content exists in the knowledge base. This is handled at the knowledge layer rather than requiring the application to implement permission filtering separately.

The full guide to connecting agents to Foundry IQ knowledge bases, including the current API version requirements and role assignments, is in Microsoft Learn: Connect Agents to Foundry IQ Knowledge Bases

Querying the knowledge base directly

Outside of a Foundry Agent context, knowledge bases can also be queried directly through the Azure AI Search REST API or SDK. This is useful for building custom application layers that need grounded retrieval without using Foundry Agent Service.

from azure.search.documents import SearchClient

search_client = SearchClient(
    endpoint=search_endpoint,
    index_name="company-documents",
    credential=credential,
    api_version="2026-05-01-preview",
)

results = search_client.knowledge_base_retrieve(
    knowledge_base_name="company-docs-kb",
    query="What is the expense reimbursement policy for travel?",
    effort="standard",
    top=5,
)

for result in results:
    print(f"Source: {result['metadata_source']}")
    print(f"Content: {result['content']}")
    print(f"Score: {result['@search.score']}")
    print("---")

The effort parameter controls the reasoning tier applied to the query. Available values depend on which API preview version is being used, with standard being the baseline tier available across versions.

When to use Foundry IQ versus building custom RAG

Foundry IQ makes most sense when the same document corpus needs to serve multiple agents, when permission enforcement at the knowledge layer is a requirement, when the team doesn't want to own the chunking, embedding, and indexer infrastructure, or when the project is already built on Microsoft Foundry and Azure AI Search.

A custom RAG implementation built directly on Azure AI Search makes more sense when the retrieval logic needs behavior that Foundry IQ doesn't expose as a configurable parameter, when the knowledge base abstraction adds more friction than it removes for a single-agent use case, or when specific preview features needed for the project might change in ways that would break a production deployment before reaching GA.

The two approaches aren't mutually exclusive. The underlying Azure AI Search service is the same in both cases, which means migrating from a custom search index to a Foundry IQ knowledge base that wraps the same index is a configuration change rather than a data migration.

References

Information based on official Microsoft documentation and verified sources as of September 2026. Foundry IQ features are actively evolving and some capabilities described may move from preview to general availability or change behavior between API versions. Always verify current status at Microsoft Learn before building production systems.