Local LLM Observability: Setting Up Langfuse for Complete Trace Visibility

By Michael Mifsud22 July 2026

#ai-engineering#observability#docker-compose#langfuse#software-engineering

The Observability Blindspot

In traditional software systems, a failure produces a stack trace, a distinct log level, or an uncaught exception. You can monitor request durations, database query execution times, and memory allocation curves to understand system health.

In LLM applications, failures are silent.

An LLM can return a response in 200 milliseconds with an HTTP 200 status code, but the output itself might be an hallucination, a broken JSON object, or a brand-damaging tone. If your application executes a complex pipeline—such as retrieving chunks from a vector database, classifying user intent, routing to a model, formatting the response, and validating the schema—monitoring only the outer boundary of the pipeline is insufficient.

When a customer reports that “the AI gave a bad answer,” you need to know:

Which exact system prompt version was active?
What text chunks were retrieved from the vector database?
Which model was routed to, and what was its temperature?
How many input and output tokens were consumed, and what was the exact cost of that single trace?

Without a dedicated LLM observability platform, you are flying blind. You are forced to dump print statements into terminal logs or maintain proprietary, complex logging abstractions that fail to scale.

Bootstrapping a Local Langfuse Stack

To understand your LLM applications, you should establish a local observability stack from day one. Self-hosting Langfuse is the most direct way to get complete, unmetered access to trace storage without sending sensitive customer data to a third-party SaaS during active development.

1. The Local Environment

Langfuse runs as a modern Next.js server backed by PostgreSQL. The entire stack can be instantiated locally using Docker Compose in under two minutes.

Create a directory on your machine, and define the following docker-compose.yml file:

version: '3.8'

services:
  postgres:
    image: postgres:16-alpine
    container_name: langfuse-postgres
    restart: always
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: local_dev_secret_password
      POSTGRES_DB: langfuse
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d langfuse"]
      interval: 5s
      timeout: 5s
      retries: 5

  langfuse:
    image: langfuse/langfuse:2
    container_name: langfuse-server
    restart: always
    depends_on:
      postgres:
        condition: service_healthy
    ports:
      - "3000:3000"
    environment:
      - DATABASE_URL=postgresql://postgres:local_dev_secret_password@postgres:5432/langfuse
      - NEXTAUTH_SECRET=local_nextauth_secret_token_12345
      - NEXTAUTH_URL=http://localhost:3000
      - TELEMETRY_ENABLED=false
      - LANGFUSE_INIT_PROJECT_NAME=Default
      - LANGFUSE_INIT_ORG_NAME=MikeMifsudEngineering
    volumes:
      - langfuse-data:/app/public/uploads

volumes:
  pgdata:
  langfuse-data:

2. Spinning Up the Services

Run the stack in detached mode:

docker compose up -d

Verify that both containers are active:

docker compose ps

Once active, open your browser and navigate to http://localhost:3000.

Register a new administrative account.
Under your Project settings, generate a new set of API Credentials:
- Host: http://localhost:3000
- Public Key: pk-lf-...
- Secret Key: sk-lf-...

Add these keys to your LLM application’s local .env configuration:

LANGFUSE_HOST="http://localhost:3000"
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_SECRET_KEY="sk-lf-..."
OPENAI_API_KEY="sk-proj-..."

Instrumenting Your Code

One of the design advantages of Langfuse is its non-invasive instrumentation. You do not have to write custom wrapping code or pass logging contexts manually down through your deep call stack. Instead, Langfuse leverages native Python decorators and automatically extracts call trees under the hood.

1. Fine-Grained Function Decorators

To capture a complete execution path, you must trace every distinct function that executes an LLM call or performs a retrieval. You should avoid simply tracing the outer boundary of your pipeline; instead, instrument each critical node.

Here is a complete, runnable example using Python and the native OpenAI client, showing how the @observe() decorator maps nested calls into a single, clean trace:

import os
from dotenv import load_dotenv
from openai import OpenAI
from langfuse.decorators import observe, langfuse_context

# Load credentials from .env
load_dotenv()

# Initialize standard clients
openai_client = OpenAI()

@observe(as_type="generation")
def classify_intent(user_message: str) -> str:
    """Classifies incoming user intent using a fast, cheap model."""
    system_prompt = "Classify the input as either: BILLING, TECHNICAL, or GENERAL."
    
    # The Langfuse decorator automatically wraps native OpenAI calls
    response = openai_client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_message}
        ],
        temperature=0.0
    )
    
    result = response.choices[0].message.content.strip()
    
    # Attach custom metadata directly to this specific generation
    langfuse_context.update_current_observation(
        output=result,
        metadata={"tokens_used": response.usage.total_tokens}
    )
    return result

@observe(as_type="generation")
def generate_support_response(category: str, query: str) -> str:
    """Generates a contextual response based on the classification category."""
    system_prompt = f"You are a helpful customer assistant. Address this {category} query."
    
    response = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": query}
        ],
        temperature=0.7
    )
    return response.choices[0].message.content.strip()

@observe()
def run_support_pipeline(user_query: str) -> str:
    """The root orchestrator executing the multi-step support workflow."""
    # Annotate the root trace with searchable tags and user IDs
    langfuse_context.update_current_trace(
        name="customer-support-routing",
        user_id="user_98234",
        tags=["production-billing", "v1.2.0"]
    )
    
    # Nested decorated calls are automatically captured as child spans!
    category = classify_intent(user_query)
    response = generate_support_response(category, user_query)
    
    return response

if __name__ == "__main__":
    test_query = "Can you help me understand why my credit card was charged twice on July 14th?"
    print("Executing pipeline...")
    final_output = run_support_pipeline(test_query)
    print(f"\nFinal Response:\n{final_output}")

2. Trace Architecture Breakdown

When the orchestrator run_support_pipeline() is executed, the @observe() decorators create a beautifully organized hierarchical trace inside your local Langfuse UI.

Inside this trace, you can inspect:

The Root Span: Shows the total duration of the entire user interaction, the overall API cost, and global tags.
Generation 1 (classify_intent): Shows the cheap gpt-4o-mini call, the system/user prompts, and the output classification (BILLING).
Generation 2 (generate_support_response): Shows the more powerful gpt-4o call executing only after Generation 1 completed, referencing the outputs generated by the previous step.

Structured Sessions: Tracking Conversations Over Time

Single traces are perfect for understanding isolated transactions, but actual applications contain continuous multi-step conversations (chatbots, multi-turn task agents, iterative debugging sessions).

To track these long-term user interactions, Langfuse provides Sessions.

By passing a consistent session_id to your root trace, you can group multiple independent traces into a single, cohesive timeline:

@observe()
def handle_chat_turn(session_id: str, user_input: str) -> str:
    # Associate this trace with a continuous session thread
    langfuse_context.update_current_trace(
        session_id=session_id,
        user_id="user_98234"
    )
    
    # Execute pipeline logic...
    return "Response content."

When you open a session page in the UI, you can view the entire chronological history of the conversation, inspect how context and memory evolved across separate turns, and quickly pinpoint exactly which turn triggered an error or a high latency spike.

Production & Security Considerations

Once your local development loop is mature, transitioning to a production deployment of Langfuse requires a shift from convenience-focused dev defaults to rigorous engineering constraints.

1. Deployment Environments

Langfuse is package-independent and stateless, making it highly compatible with modern infrastructure. Any platform capable of hosting Docker containers can run it:

PaaS Fallbacks: Platforms like Railway, Render, or Heroku are excellent for quick, low-maintenance staging environments.
Cloud Infrastructure: For production, host the Next.js server on AWS ECS/Fargate, Google Cloud Run, or Azure Container Apps, backed by a fully managed, auto-scaling database service like AWS RDS Aurora PostgreSQL.

2. Hardening Your Configuration

When moving to production, you must explicitly disable development behaviors:

Enforce HTTPS: Ensure all endpoints run behind an SSL/TLS terminator.
Enable OAuth: Do not rely on simple email/password credentials. Configure official identity providers (e.g., GitHub, Google, Okta, or generic OIDC) by setting AUTH_GOOGLE_CLIENT_ID and AUTH_GOOGLE_CLIENT_SECRET.
Database Isolation: Never expose your PostgreSQL port (5432) to the public internet. Ensure the database resides exclusively within an isolated private VPC subnet, reachable only by the Langfuse application servers.
Aggressive Secret Management: Ensure production API keys, NextAuth secret strings, and DB credentials are never checked into git repos. Use secure environment managers (e.g., AWS Secrets Manager, GCP Secret Manager, or HashiCorp Vault) and map them dynamically during runtime.

Observability as an Architectural Principle

Observability is not something you should retroactively stitch onto a broken application after customers start complaining. It should be treated as a core architectural constraint of your system design.

By setting up local self-hosted tracing with Langfuse and integrating decorators into your core function schemas, you establish a permanent feedback loop. The detailed execution traces, latency markers, token usage models, and session histories captured during development are the exact inputs required to write programmatic unit evals, construct strong expert judge datasets, and scale your AI applications safely and predictably.