Enterprise AI Patterns for .NET Developers: 12 Patterns I Wish I Had on Day One

Two years ago I reviewed a pull request that made me genuinely uncomfortable. A senior developer on our team, someone I respected, had wired an LLM call directly into a controller action. Raw HttpClient, a hardcoded prompt string concatenated with user input, the response parsed with Substring and IndexOf, no retry, no timeout, no logging of what the model actually said. It worked in the demo. It was also everything we spent fifteen years learning not to do with databases and message brokers, done all over again with a new dependency that happens to be non-deterministic.

That PR was not an outlier. It was the default. When a technology is new, we forget our own discipline. We treated SQL like that in 2005, HTTP APIs like that in 2012, and now LLMs like that in the mid 2020s.

The good news is that the .NET ecosystem has matured dramatically. Microsoft.Extensions.AI gave us a standard abstraction layer, and Microsoft Agent Framework hit 1.0 in April 2026, merging Semantic Kernel and AutoGen into one supported platform with stable APIs. The building blocks exist. What is still missing in most codebases I see is the architectural discipline: the patterns that turn “we call an LLM” into “we run an AI system in production.”

This article is my attempt to write down the twelve patterns I keep reaching for. Most of them I learned the hard way, on systems where the model misbehaved at 2 AM and the pattern was the only thing standing between an incident and a non-event. All examples are in C#, and every cloud-dependent example has a local Ollama alternative, because I refuse to pay per token to run unit tests.

The Local Development Baseline

Before the patterns, the setup. Everything in this article runs against Ollama locally, and the same code runs against Azure OpenAI, OpenAI, Anthropic, or Bedrock in production because we never touch a provider SDK directly. That is Pattern 1 doing its job before we even name it.

# Install Ollama, then pull the models used throughout this article
ollama pull llama3.1:8b
ollama pull nomic-embed-text
ollama serve

dotnet add package Microsoft.Extensions.AI
dotnet add package OllamaSharp

using Microsoft.Extensions.AI;
using OllamaSharp;

// OllamaApiClient implements IChatClient directly
IChatClient local = new OllamaApiClient(
    new Uri("http://localhost:11434"), "llama3.1:8b");
var response = await local.GetResponseAsync("Say hello in one word.");
Console.WriteLine(response.Text);

Five lines, no API key, no bill. Every pattern below builds on this.

Pattern 1: The Chat Client Abstraction

The single most important decision in an enterprise AI codebase is that no business code ever references a provider SDK. Not OpenAIClient, not AmazonBedrockRuntimeClient, not OllamaApiClient. Business code sees IChatClient and nothing else.

+------------------+ +--------------------+ +------------------+
| Business Logic | --> | IChatClient | --> | Provider Impl |
| (no SDK refs) | | (abstraction) | | (swappable) |
+------------------+ +--------------------+ +------------------+
                                                      |
                                                      +-- Azure OpenAI
                                                      +-- OpenAI
                                                      +-- Anthropic
                                                      +-- Bedrock
                                                      +-- Ollama (dev)

The environment decides the implementation, dependency injection delivers it:

builder.Services.AddChatClient(services =>
    builder.Environment.IsDevelopment()
        ? new OllamaApiClient(new Uri("http://localhost:11434"), "llama3.1:8b")
        : new AzureOpenAIClient(
              new Uri(config["AzureOpenAI:Endpoint"]!),
              new DefaultAzureCredential())
          .GetChatClient(config["AzureOpenAI:Deployment"]!)
          .AsIChatClient());

I have swapped providers three times on one system without touching a single line of business logic. If you take only one thing from this article, take this pattern. Everything else depends on it.

Pattern 2: The Middleware Pipeline

Once everything is an IChatClient, cross-cutting concerns become decorators. This is the same mental model as ASP.NET Core middleware, and ChatClientBuilder makes it explicit:

builder.Services.AddChatClient(services =>
    new ChatClientBuilder(innerClient)
        .UseDistributedCache() // outermost: check cache first
        .UseFunctionInvocation() // auto tool-call loop
        .UseOpenTelemetry() // spans for every model call
        .UseLogging()
        .Build());

Request flow through the pipeline:
caller
    |
    v
+----------+ +-----------+ +-----------+ +---------+ +-------+
| Cache |-->| Function |-->| OTel |-->| Logging |-->| Model |
| check | | invocation| | tracing | | | | |
+----------+ +-----------+ +-----------+ +---------+ +-------+
    | |
    +---- cache hit: short-circuit, model never called <---------+

The discipline here is refusing to write these concerns inline. Every time I see a try/catch with manual logging wrapped around a model call inside a service method, I know the codebase has five slightly different versions of that block. Middleware means one version, tested once.

Pattern 3: Structured Output as a Contract

Free-form text is a demo format, not an integration format. In production, the model’s output is an API response, and API responses have schemas. I treat every model interaction that feeds downstream code as a typed contract:

public record InvoiceExtraction(
    string VendorName,
    decimal TotalAmount,
    string Currency,
    DateOnly? DueDate,
    string[] LineItemDescriptions);
var result = await chatClient.GetResponseAsync<InvoiceExtraction>(
    $"Extract the invoice fields from this document:\n{documentText}");
InvoiceExtraction invoice = result.Result; // typed, validated, done

GetResponseAsync in Microsoft.Extensions.AI generates the JSON schema from the type and handles deserialization. With Ollama this works surprisingly well on llama3.1:8b for flat records, though nested structures need a bigger model or a retry loop.

The part most teams skip: validate after deserialization. A schema guarantees shape, not sense.

if (invoice.TotalAmount < 0 || invoice.TotalAmount > 1_000_000m)
    throw new ExtractionOutOfRangeException(invoice);

The model can produce a perfectly valid JSON document that says an invoice is for negative four million euros. Schemas do not save you from that. FluentValidation does.

Pattern 4: Typed Tool Calling

Tool calling is where LLMs stop being text generators and start being system participants. The pattern that matters is treating tools as normal, testable C# methods with descriptions, never as prompt-embedded instructions:

[Description("Gets the current stock level for a product SKU")]
static async Task<int> GetStockLevel(
    [Description("The product SKU, e.g. WH-1000")] string sku,
    IInventoryService inventory)
    => await inventory.GetAvailableAsync(sku);
var options = new ChatOptions
{
    Tools = [AIFunctionFactory.Create(GetStockLevel)]
};

Tool call loop (handled by UseFunctionInvocation):
user question
       |
       v
  +---------+ tool_call +--------------+
  | Model | -------------> | GetStockLevel|
  | | <------------- | (your code) |
  +---------+ result: 42 +--------------+
       |
       v
  "There are 42 units of WH-1000 in stock."

Two rules I enforce in review. First, tools are idempotent reads by default; anything that writes goes through Pattern 9 (human-in-the-loop) or an explicit allow list. Second, tool implementations get unit tests like any other code, because they are any other code. The model is just an unusual caller.

Pattern 5: RAG as a Query Pipeline, Not a Feature

Retrieval-augmented generation gets sold as a product feature. Architecturally it is a query pipeline with four stages, and each stage is independently replaceable:

+--------+ +-----------+ +-----------+ +----------+ +--------+
| Ingest |-->| Embed |-->| Store |-->| Retrieve |-->| Answer |
| chunk | | (vectors) | | (pgvector)| | (top-k + | | (LLM + |
| docs | | | | | | rerank) | | context)|
+--------+ +-----------+ +-----------+ +----------+ +--------+

The IEmbeddingGenerator abstraction plays the same role for embeddings that IChatClient plays for chat:

IEmbeddingGenerator<string, Embedding<float>> embedder =
    new OllamaApiClient(new Uri("http://localhost:11434"), "nomic-embed-text");
var embedding = await embedder.GenerateAsync("refund policy for damaged goods");
float[] vector = embedding.Vector.ToArray();

For storage I default to PostgreSQL with pgvector rather than a dedicated vector database, because in an enterprise .NET shop you already run Postgres, you already back it up, and your DBA already trusts it:

CREATE TABLE doc_chunks (
    id bigserial PRIMARY KEY,
    content text NOT NULL,
    embedding vector(768),
    source_uri text NOT NULL,
    updated_at timestamptz NOT NULL DEFAULT now()
);

CREATE INDEX ON doc_chunks USING hnsw (embedding vector_cosine_ops);

The mistake I made on my first RAG system was treating retrieval quality as a prompt problem. It is a data problem. Bad chunking, stale documents, and missing metadata filters caused ninety percent of our bad answers. The model was fine. The pipeline feeding it was not.

Pattern 6: Semantic Caching

Exact-match caching (UseDistributedCache) only helps when inputs repeat verbatim. In real systems, users ask the same question a hundred slightly different ways. Semantic caching embeds the query, searches for a previously answered near-duplicate, and returns the stored answer above a similarity threshold:

incoming query
       |
       v
  embed query --> cosine search in answer cache
       |
       +-- similarity >= 0.95 --> return cached answer (cost: 1 embedding)
       |
       +-- similarity < 0.95 --> call model, store (query, answer, vector)

public async Task<string> GetAnswerAsync(string query)
{
    var queryVec = (await _embedder.GenerateAsync(query)).Vector;
var hit = await _cacheStore.FindNearestAsync(queryVec, minSimilarity: 0.95f);
    if (hit is not null)
        return hit.Answer;
    var response = await _chatClient.GetResponseAsync(query);
    await _cacheStore.SaveAsync(query, queryVec, response.Text);
    return response.Text;
}

On one internal support assistant this cut model spend by roughly forty percent. But I will be honest about the sharp edge: the threshold is a business decision disguised as a number. At 0.95 you get safe but modest hit rates. At 0.90 you occasionally serve an answer to a question the user did not quite ask, and those failures are embarrassing in a way a slow response never is. Start conservative, measure, and never semantically cache anything personalized or time-sensitive.

Pattern 7: Resilience and Model Fallback Routing

LLM providers throttle, degrade, and go down. Treating a model endpoint as more reliable than any other remote dependency is wishful thinking. I wrap model calls with Polly the same way I wrap payment gateways, with one AI-specific addition: fallback is not just retry, it is rerouting to a different model:

                 +----------------------+
   request --> | Primary: gpt-4.1 |
                 +----------------------+
                       | 429 / timeout / 5xx
                       v
                 +----------------------+
                 | Retry x2, backoff |
                 +----------------------+
                       | still failing
                       v
                 +----------------------+
                 | Fallback: smaller |
                 | model, degraded but |
                 | answering |
                 +----------------------+
                       | circuit open
                       v
                 +----------------------+
                 | Static response + |
                 | queue for later |
                 +----------------------+

var pipeline = new ResiliencePipelineBuilder<ChatResponse>()
    .AddRetry(new()
    {
        MaxRetryAttempts = 2,
        BackoffType = DelayBackoffType.Exponential,
        ShouldHandle = new PredicateBuilder<ChatResponse>()
            .Handle<HttpRequestException>()
            .Handle<TaskCanceledException>()
    })
    .AddFallback(new()
    {
        FallbackAction = _ => Outcome.FromResultAsValueTask(
            await _fallbackClient.GetResponseAsync(messages))
    })
    .AddCircuitBreaker(new() { FailureRatio = 0.5, BreakDuration = TimeSpan.FromSeconds(30) })
    .Build();

The subtle point: the fallback model produces different output quality, and downstream consumers must tolerate that. I tag every response with which model produced it, both in telemetry and, for some workflows, in the payload itself. When quality complaints come in, the first question is always “which model answered this,” and you want that answer in one query, not one archaeology session.

Pattern 8: The Verification Layer

This is the pattern I care about most, and the one I have built more than once. The idea: no model output reaches a user or a downstream system without passing through verification, and verification is layered from cheap to expensive:

model output
     |
     v
+---------------------------+
| L1: Deterministic checks | schema, ranges, regex, allow lists
| (microseconds, free) | business rule assertions
+---------------------------+
     | pass
     v
+---------------------------+
| L2: Grounding checks | do cited sources exist?
| (milliseconds) | do quoted numbers appear in context?
+---------------------------+
     | pass
     v
+---------------------------+
| L3: LLM-as-judge | a second model scores faithfulness
| (sampled, expensive) | run on 100% high-risk, 5% sampled
+---------------------------+
     | pass
     v
  deliver

Layer 1 catches most failures for almost no cost. A model that invents an order status not in your enum, a date in 1970, a refund above policy limits: all of that dies in deterministic checks. Layer 3, the LLM-as-judge, is powerful but expensive and itself fallible, so I run it on every high-risk action and a sample of everything else, feeding the scores into the evaluation loop (Pattern 12).

public async Task<VerificationResult> VerifyAsync(
    AgentAnswer answer, RetrievalContext context)
{
    // L1: deterministic
    var l1 = _ruleEngine.Check(answer);
    if (!l1.Passed) return VerificationResult.Rejected(l1);
// L2: grounding, every cited chunk id must exist in the context
    var unknownCitations = answer.CitedChunkIds.Except(context.ChunkIds).ToList();
    if (unknownCitations.Count > 0)
        return VerificationResult.Rejected($"Unknown citations: {string.Join(",", unknownCitations)}");
    // L3: judge, sampled
    if (answer.RiskTier == RiskTier.High || _sampler.ShouldSample())
    {
        var verdict = await _judgeClient.GetResponseAsync<JudgeVerdict>(
            JudgePrompt.For(answer, context));
        if (verdict.Result.FaithfulnessScore < 0.7)
            return VerificationResult.Escalated(verdict.Result);
    }
    return VerificationResult.Approved();
}

For local development, the judge runs on llama3.1:8b via the same IChatClient abstraction. A local judge is noticeably less consistent than a frontier model, but for wiring and testing the escalation paths it is exactly what you need.

Pattern 9: Human-in-the-Loop Approval Gates

Any action with real-world consequences (sending money, emailing customers, changing records) goes through an approval gate. The architectural insight is that this is not an AI pattern at all. It is a workflow pattern: the agent proposes, the proposal is persisted, a human disposes, and the workflow resumes.

agent decides "issue refund of 240 EUR"
     |
     v
+---------------------+ +--------------+ +--------------------+
| Persist proposal | --> | Notify human | --> | Human approves or |
| (status: pending) | | (Slack/queue)| | rejects in UI |
+---------------------+ +--------------+ +--------------------+
                                                       |
                              +------------------------+
                              v
                    +--------------------+
                    | Resume workflow |
                    | with decision |
                    +--------------------+

The implementation detail that matters in .NET: the wait can last hours or days, so the state lives in a database, not in memory, and resumption is triggered by an event. Microsoft Agent Framework 1.0 ships human-in-the-loop approval flows as a first-class harness feature, which validates what many of us were hand-rolling. Whether you use MAF’s built-in support or your own table plus a MassTransit saga, the invariant is the same: the agent process must be safely killable while a proposal is pending, and the decision must be auditable forever.

One rule from experience: approvals must carry full context. An approval request that says “Approve refund? Y/N” trains humans to click yes. One that shows the customer history, the agent’s reasoning, and the verification scores from Pattern 8 lets humans actually judge. A rubber-stamp gate is worse than no gate, because it produces false confidence.

Pattern 10: Event-Driven Agent Triggers with the Outbox

Most enterprise AI work is not a chat window. It is “when an invoice arrives, extract and validate it,” “when a ticket is created, triage it.” Agents are event consumers, and everything we know about event-driven .NET applies, especially the transactional outbox:

+-------------+ +-----------------+ +----------------+
| Business tx | | Outbox table | | Broker |
| (EF Core) |--->| (same db tx) |--->| (RabbitMQ / |
| | | | | Azure SB) |
+-------------+ +-----------------+ +----------------+
                                              |
                                              v
                                     +-----------------+
                                     | Agent consumer |
                                     | (MassTransit) |
                                     | idempotent! |
                                     +-----------------+

public class InvoiceReceivedConsumer : IConsumer<InvoiceReceived>
{
    public async Task Consume(ConsumeContext<InvoiceReceived> ctx)
    {
        // Idempotency first: model calls are expensive and non-deterministic,
        // reprocessing the same event must not produce a second extraction
        if (await _store.AlreadyProcessedAsync(ctx.Message.InvoiceId))
            return;
        var extraction = await _extractionAgent.RunAsync(ctx.Message.DocumentUri);
        await _store.SaveAsync(ctx.Message.InvoiceId, extraction);
        await ctx.Publish(new InvoiceExtracted(ctx.Message.InvoiceId));
    }
}

Idempotency deserves emphasis. With a deterministic consumer, redelivery produces the same result twice, which is wasteful but harmless. With an LLM consumer, redelivery produces a different result the second time, which can mean two conflicting extractions of the same invoice in your database. Deduplicate on the way in, always.

Pattern 11: Explicit Multi-Agent Orchestration

When one agent grows too many responsibilities, the temptation is to let agents talk freely to each other. Resist it. Free-form agent conversation is impossible to debug, cost-bound, or test. The pattern that survives production is an explicit graph: an orchestrator routes work to specialists, and the topology is code, not emergent behavior.

                    +-----------------+
                    | Orchestrator |
                    | (routing only) |
                    +-----------------+
                     / | \
                    v v v
            +---------+ +---------+ +----------+
            | Triage | | Research| | Drafting |
            | agent | | agent | | agent |
            +---------+ +---------+ +----------+
                    \ | /
                     v v v
                    +-----------------+
                    | Verification |
                    | (Pattern 8) |
                    +-----------------+

This is where Microsoft Agent Framework earns its place: it models multi-agent workflows as explicit graphs with typed edges, the direct successor to what Semantic Kernel and AutoGen each did partially. I built the same shape on LangGraph in Python for other systems, and the convergence is striking. Everyone who runs multi-agent systems in production ends up at the same place: deterministic graph, non-deterministic nodes.

My rule of thumb for when to split into multiple agents: when a single agent’s system prompt starts containing paragraphs that begin with “unless” and “except when,” the prompt is telling you it wants to be two agents.

Pattern 12: Observability and the Evaluation Loop

The last pattern closes the loop. Traditional monitoring answers “is it up.” AI systems need a second question answered continuously: “is it still good.” Those are different pipelines with different tools, and both are non-negotiable:

+--------------------------------------------------------------+
| Production traffic |
+--------------------------------------------------------------+
      | |
      v v
+------------------+ +-------------------------+
| Telemetry | | Eval pipeline |
| OTel spans: | | golden dataset (CI) |
| tokens, latency, | | sampled prod traces |
| model id, cost | | judge scores (P8, L3) |
+------------------+ +-------------------------+
      | |
      v v
+------------------+ +-------------------------+
| Dashboards and | | Regression gate: |
| cost alerts | | block deploy if scores |
+------------------+ | drop below baseline |
                                     +-------------------------+

The telemetry half is nearly free with Microsoft.Extensions.AI: .UseOpenTelemetry() on the client builder emits spans per model call following the GenAI semantic conventions, and they land in whatever OTel backend you already run.

The evaluation half is the part teams postpone and regret. The minimum viable version is a golden dataset of fifty real cases with expected outcomes, run in CI against the same code path as production:

[Theory]
[MemberData(nameof(GoldenInvoices))]
public async Task Extraction_matches_golden_expectations(GoldenCase c)
{
    var result = await _extractionAgent.RunAsync(c.DocumentUri);
    Assert.Equal(c.Expected.TotalAmount, result.TotalAmount);
    Assert.Equal(c.Expected.Currency, result.Currency);
    // Fuzzy fields get scored, not asserted
    var score = await _judge.ScoreFieldAsync(c.Expected.VendorName, result.VendorName);
    Assert.True(score >= 0.8, $"Vendor name drifted: {result.VendorName}");
}

Run this suite against Ollama on every PR for fast, free signal, and against the production model nightly. When a provider silently updates a model, and they do, this suite is how you find out before your users do.

Production Reality Check

I want to be honest about the limits of everything above, because pattern articles have a way of implying that architecture solves the problem. It does not. It contains the problem.

The verification layer (Pattern 8) reduces bad outputs; it does not eliminate them. My LLM-as-judge disagrees with human reviewers roughly one time in ten, and I still do not have a principled way to set the faithfulness threshold beyond “tune it until the escalation queue is manageable.”

Semantic caching (Pattern 6) has served a subtly wrong answer to production users. Twice that I know of. The similarity threshold that prevents this also cuts your hit rate in half, and there is no free lunch there.

The local Ollama story is genuinely excellent for wiring, integration tests, and CI, but llama3.1:8b is not a proxy for frontier model behavior on tool selection or complex structured output. Tests that pass locally and fail against the production model are a real category, which is exactly why the nightly eval run against the real model exists.

And multi-agent orchestration (Pattern 11) multiplies cost and latency faster than it multiplies capability. My default is still one agent with good tools, and I split only when the prompt forces me to. The pattern exists for when you need it, not as a starting point.

Closing Thoughts

None of these twelve patterns is exotic. Abstraction layers, middleware, contracts, caching, resilience, verification, approval workflows, outboxes, explicit orchestration, observability: this is the same discipline .NET developers have applied to every other unreliable dependency for two decades. The only genuinely new ingredient is non-determinism, and the honest summary of this entire article is one sentence: treat the model as a brilliant, fast, occasionally wrong remote service, and wrap it in everything you would wrap around a service you do not fully trust. Because you should not fully trust it.

The ecosystem finally supports this discipline. Microsoft.Extensions.AI gives you the abstraction and the middleware pipeline. Agent Framework 1.0 gives you stable, supported orchestration with human-in-the-loop and observability built in. Ollama gives you a zero-cost local loop. The tooling excuse is gone. What remains is the engineering.

If you build one thing after reading this, build the verification layer. It is the pattern that converts “the model said something wrong” from an incident into a log line.

Other Articles

• Workflow Design Is a Thinking Discipline
• Event-Driven Systems in .NET, Python, and Go: A Practitioner’s Comparison
• The Verification Layer Every AI Agent Needs (and How I Built One Twice)

Tags : dotnet, artificial-intelligence, software-architecture, agentic-ai, csharp, llm, enterprise-software, microsoft, software-engineering, machine-learning