An agent makes six tool calls, picks the wrong one on step four, and the final output is garbage. You stare at your logs. You see the input. You see the output. Everything in between is a void. That's the black box problem with agentic LLM workflows, and it's the reason I started looking at Langfuse seriously.
If you're running multi-step agents (LangChain, custom loops, or any orchestration layer), you need per-step tracing with enough context to reconstruct why the agent chose what it chose. Langfuse gives you that. But getting it wired up correctly, especially in a self-hosted Kubernetes environment alongside other observability tools, has a few sharp edges worth knowing about.
Observability Sprawl: The Failure Mode Nobody Talks About
Before I get into Langfuse itself, I want to talk about a failure mode I see constantly with LLM tooling: observability sprawl.
Here's how it usually plays out. You spin up Dify because it has a nice agent builder. You add Opik because someone recommended it for evaluation. You deploy AnythingLLM for RAG experiments. Each tool has its own Postgres database, its own PVC, its own memory footprint. Before you know it, you've got three separate platforms that each capture some traces, and none of them give you the full picture.
Resource costs compound quickly. In a homelab or small-cluster environment, those redundant tools can easily consume 8-10 GB of RAM and 50-70 GB of persistent storage. Those are real resources you're giving up for the privilege of having your debugging split across multiple dashboards.
Pick one tool, instrument everything through it, and delete the rest. Langfuse is the one I picked, and the consolidation alone was worth it. But the reason I picked it over alternatives comes down to one specific feature.
What I Actually Needed (And What Most Tools Get Wrong)
Most LLM observability tools trace at the wrong granularity. They capture the top-level call: here's the prompt, here's the completion, here's the token count. Fine for a single chat.completions call. Nearly useless for an agentic workflow.
Consider what happens in a typical agent loop. An orchestrator receives a user query. It decides which tool to call. That tool might call an LLM itself (for summarization, extraction, or routing). Results come back, and the orchestrator decides whether to call another tool or return a final answer. A single user request might involve four or five LLM calls, each with different prompts, different models, and different failure modes.
What I needed was the ability to trace the full execution tree: one top-level "trace" for the user request, with nested "spans" for each agent step, and nested "generations" for each LLM call within those steps. Langfuse calls this the trace/span/generation hierarchy, and it maps cleanly onto how multi-agent systems actually work.
Evaluation scores were the other hard requirement, and I wanted them attached to traces, not living in a separate system. I had a custom evaluation layer built with Zod schemas that validated agent outputs against expected structures. It worked, but it was brittle, lived in application code, and had no dashboard. Langfuse lets you attach numeric scores to any trace or span, which means your evaluation data lives right next to your trace data. One place to look.
Setting Up Langfuse (Self-Hosted on Kubernetes)
Langfuse has a managed cloud offering, but if you're already running a cluster, self-hosting is straightforward. The project provides a Helm chart and Docker images. The main dependency is Postgres.
If you're already running CloudNativePG, you can point Langfuse at an existing cluster. Create a dedicated database for it:
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: langfuse-db
namespace: observability
spec:
instances: 2
storage:
size: 10Gi
bootstrap:
initdb:
database: langfuse
owner: langfuse
Langfuse itself is a single container with environment variables for the database connection, a secret key, and your desired auth settings. A minimal Kubernetes deployment looks like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: langfuse
namespace: observability
spec:
replicas: 1
selector:
matchLabels:
app: langfuse
template:
spec:
containers:
- name: langfuse
image: langfuse/langfuse:2.x
ports:
- containerPort: 3000
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: langfuse-db-credentials
key: uri
- name: NEXTAUTH_SECRET
valueFrom:
secretKeyRef:
name: langfuse-auth
key: secret
- name: NEXTAUTH_URL
value: "https://langfuse.example.com"
- name: SALT
valueFrom:
secretKeyRef:
name: langfuse-auth
key: salt
If you're deploying through ArgoCD, there's a gotcha worth flagging. If you organize your observability stack in a directory structure (say, observability/langfuse/, observability/grafana/, etc.) and use a directory-type Application source, you need to set directory.recurse: true. Without it, ArgoCD will show "0 managed resources" even though your manifests exist in subdirectories. It's a silent failure that'll have you rechecking file paths for twenty minutes before you realize ArgoCD just isn't looking deep enough.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: observability
spec:
source:
repoURL: https://git.example.com/infra.git
path: observability
directory:
recurse: true # without this, subdirectories are invisible
Instrumenting Agent Workflows
Once Langfuse is running, the real work begins: instrumenting your agent code so each step shows up as a distinct span in the trace tree. Langfuse's Python SDK makes this fairly clean with decorators.
A minimal example for a custom agent loop:
from langfuse.decorators import observe, langfuse_context
from openai import OpenAI
client = OpenAI()
@observe(as_type="generation")
def call_llm(prompt: str, model: str = "gpt-4o") -> str:
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
)
return response.choices[0].message.content
@observe()
def search_tool(query: str) -> str:
# your tool logic here
results = do_search(query)
return results
@observe()
def agent_loop(user_query: str) -> str:
plan = call_llm(f"Plan steps for: {user_query}")
for step in parse_steps(plan):
if step.tool == "search":
result = search_tool(step.input)
elif step.tool == "summarize":
result = call_llm(f"Summarize: {step.input}")
# each iteration creates a child span automatically
final = call_llm(f"Final answer given results: {result}")
return final
Every function decorated with @observe() becomes a span in Langfuse. Functions marked as_type="generation" get special treatment: Langfuse records token counts, model name, latency, and prompt/completion pairs. Nested calls automatically create a parent-child hierarchy, so when you open a trace in the Langfuse UI, you see the full tree.
For TypeScript/Node.js backends, the pattern is similar but uses the Langfuse client class directly:
import Langfuse from "langfuse";
const langfuse = new Langfuse({
publicKey: process.env.LANGFUSE_PUBLIC_KEY,
secretKey: process.env.LANGFUSE_SECRET_KEY,
baseUrl: "https://langfuse.example.com",
});
async function tracedAgentCall(userQuery: string) {
const trace = langfuse.trace({ name: "agent-request" });
const planSpan = trace.span({ name: "planning" });
const plan = await callLLM(userQuery);
planSpan.update({ output: plan });
planSpan.end();
for (const step of parseSteps(plan)) {
const toolSpan = trace.span({
name: `tool:${step.tool}`,
input: step.input,
});
const result = await executeTool(step);
toolSpan.update({ output: result });
toolSpan.end();
}
await langfuse.flushAsync();
}
Notice the explicit flushAsync() at the end. Langfuse batches events for performance. In serverless or short-lived processes, skipping the flush means you lose traces silently. I've seen this bite people running agents in Lambda functions or one-shot scripts.
Replacing Custom Evaluation Logic
Before Langfuse, my evaluation layer was a hand-rolled mess. Zod schemas validated agent outputs, results got persisted to a JSON file or a database table, and "evaluation" meant grepping through structured logs. It worked, in the sense that a Rube Goldberg machine works.
Langfuse replaces that with score calls attached directly to traces:
# Before: custom evaluation persisted to database
from zod_validator import AgentOutputSchema
import json
def evaluate_and_persist(output, expected_schema):
result = AgentOutputSchema.safeParse(output)
with open("eval_log.jsonl", "a") as f:
json.dump({
"valid": result.success,
"errors": result.errors if not result.success else None,
"timestamp": datetime.now().isoformat()
}, f)
f.write("\n")
# After: scores live in Langfuse alongside traces
from langfuse.decorators import observe, langfuse_context
@observe()
def agent_with_eval(user_query: str) -> str:
result = agent_loop(user_query)
# attach a quality score to this trace
langfuse_context.score_current_trace(
name="output_valid",
value=1.0 if validate_output(result) else 0.0,
)
# attach a relevance score
langfuse_context.score_current_trace(
name="relevance",
value=compute_relevance(user_query, result),
comment="cosine similarity against expected answer",
)
return result
Now your evaluation data shows up in the same dashboard as your traces. You can filter traces by score, spot regressions over time, and correlate low scores with specific agent steps that failed. No more cross-referencing JSON log files with application logs.
Why This Architecture Actually Works
Langfuse's trace/span/generation model maps onto agentic workflows because it mirrors the actual call stack. A trace is a complete user request. Spans are logical operations within that request. Generations are the individual LLM calls.
This hierarchy means you can answer questions that flat logging can't:
- "Which tool call is the bottleneck?" Sort spans by latency.
- "Why did the agent hallucinate on this request?" Open the trace, find the span where the wrong tool was selected, inspect the prompt and completion.
- "Are my evaluations degrading over time?" Filter by score name, plot the trend.
Compare this to what you get with Grafana dashboards. Grafana excels at aggregate metrics: request rate, p99 latency, error percentage. It shows you the forest. Langfuse shows you individual trees. You need both, but for debugging agent behavior, the per-trace detail is what saves you.
Prompt management is another underappreciated feature. Langfuse lets you version prompts in its UI, then fetch them at runtime by name and version. This decouples prompt iteration from code deployment. Your prompt engineer (or you, wearing that hat) can tweak prompts and track how each version affects scores, without touching application code or triggering a redeploy.
Credential Security for Agent Traces
One thing to think about early: agent traces often contain sensitive data. Tool inputs might include search queries, user data, or service account credentials. Langfuse stores everything you send it.
If you're self-hosting, this is manageable because the data stays in your cluster. But you should still be intentional about what gets logged. Scrub sensitive fields before they hit the trace:
@observe()
def safe_tool_call(tool_name: str, params: dict) -> str:
sanitized = {k: v for k, v in params.items()
if k not in ["api_key", "token", "password"]}
langfuse_context.update_current_observation(
input=sanitized, # only safe fields
)
return execute_tool(tool_name, params) # full params for execution
For managed Langfuse (their cloud), check your data handling requirements before shipping traces that contain PII or internal API responses. If you're building AI agent services for clients, this is a compliance conversation you want to have before the first trace lands.
Lessons Learned
Consolidate early. Running multiple LLM observability tools feels productive because you're "evaluating options." In practice, it means your traces are fragmented, your resource usage balloons, and you debug slower because you're checking two dashboards for every issue. Pick one tool and commit. If Langfuse doesn't fit your stack, pick something else, but pick one.
Instrument at the span level from day one. Adding tracing to an existing agent codebase after the fact is painful. Every function needs to be wrapped, and you inevitably miss the one tool call that turns out to be the problem. If you're building a new agent, add @observe() decorators as you write each function. Retrofitting is always harder.
Flush your traces. Langfuse batches events for efficiency, which means traces can be lost if your process exits before the batch ships. Call langfuse.flush() (Python) or langfuse.flushAsync() (TypeScript) at the end of every request handler. In serverless environments, this is not optional.
Scores are cheap. Use them. Attaching a score call adds negligible overhead, but it gives you trend data you can't get any other way. Even a simple binary "output was valid" score, aggregated over hundreds of traces, tells you whether your agent is getting better or worse after a prompt change. I score every trace now, even if the scoring logic is basic.
Self-hosting is worth it for sensitive workloads. Agent traces contain prompts, tool outputs, and sometimes user data. Keeping that data on your own cluster, behind your own network policies, is worth the operational overhead of managing a Postgres database and a single container. If you're already running Kubernetes with CloudNativePG, the marginal cost is low.
Langfuse isn't the most exciting tool I've deployed. It doesn't generate flashy demos. But it's the tool that made my agent debugging go from "stare at logs and guess" to "open the trace, click the failing span, read the prompt." For anything running in production, that difference is everything.
