Hermes Agent tracing with Langfuse

This notebook shows how to integrate Langfuse with Hermes Agent to trace, debug, and evaluate your agent's conversations, LLM calls, and tool usage.

What is Hermes Agent? Hermes Agent is a self-improving AI agent built by Nous Research. It features a built-in learning loop, persistent memory, autonomous skill creation, and support for any LLM provider. Hermes ships a bundled Langfuse observability plugin that traces every conversation turn, LLM request, and tool call.

What is Langfuse? Langfuse is an open-source AI engineering platform that helps teams trace, debug, and evaluate their LLM applications.

The steps below follow Hermes' official Langfuse plugin docs — refer to them for the latest details.

Step 1: Install Dependencies

%pip install git+https://github.com/NousResearch/hermes-agent.git langfuse -U

Step 2: Set Up Environment Variables

Get your Langfuse keys from the project settings in Langfuse Cloud or set up self-hosting.

Hermes reads credentials from ~/.hermes/.env (the canonical location per the Hermes docs). Create the file with:

# ~/.hermes/.env
HERMES_LANGFUSE_PUBLIC_KEY=pk-lf-...
HERMES_LANGFUSE_SECRET_KEY=sk-lf-...
HERMES_LANGFUSE_BASE_URL=https://cloud.langfuse.com   # or your self-hosted URL

The plugin also accepts the standard SDK env vars (LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL); the HERMES_LANGFUSE_* variants win when both are set.

The cell below sets the same credentials inside this Python kernel so we can quickly verify them with the Langfuse SDK. Note: these os.environ values are scoped to the notebook process and will not be visible to a hermes chat command run in a separate terminal — use ~/.hermes/.env for that.

import os

# Get keys for your project from the project settings page: https://langfuse.com/cloud
os.environ.setdefault("LANGFUSE_PUBLIC_KEY", "pk-lf-...")
os.environ.setdefault("LANGFUSE_SECRET_KEY", "sk-lf-...")
os.environ.setdefault("LANGFUSE_BASE_URL", "https://cloud.langfuse.com") # 🇪🇺 EU region

With the environment variables set, initialize the Langfuse client to confirm your credentials work. Hermes uses its own internal client, so this step is purely a sanity check that your keys are valid.

from langfuse import get_client

langfuse = get_client()

# Verify connection
if langfuse.auth_check():
    print("Langfuse client is authenticated and ready!")
else:
    print("Authentication failed. Please check your credentials and host.")

Step 3: Enable the Langfuse Plugin

Hermes ships a bundled Langfuse observability plugin under plugins/observability/langfuse. Bundled plugins are discovered automatically but opt-in — they don't load until you explicitly enable them.

The plugin hooks into Hermes lifecycle events (pre_api_request / post_api_request, pre_tool_call / post_tool_call) to automatically capture:

• One root span per conversation turn ("Hermes turn")
• One generation observation per LLM API call
• One tool observation per tool call

Session grouping uses the Hermes session ID (or task ID for sub-agents), so every turn within a hermes chat session lives under one Langfuse session. The plugin is also fail-open: missing SDK, missing credentials, or a transient Langfuse error all turn into a silent no-op — the agent loop is never impacted.

# Enable the Langfuse plugin (run this in your terminal, not in a notebook)
# hermes plugins enable observability/langfuse

Step 4: Run Hermes and Generate a Trace

With the plugin enabled and credentials set, every Hermes conversation turn is automatically traced to Langfuse. Each trace captures:

• Conversation turns as the root span ("Hermes turn")
• LLM calls as generation observations with model, usage, cost, and latency
• Tool calls as tool observations with input arguments and results
• Token usage and cost broken down by input, output, cache, and reasoning tokens

You can start a conversation from the CLI:

# Send a one-off message (traces are sent automatically):
# hermes chat -q "hello"

# Or start a full interactive session:
# hermes chat

Optional: Tune Tracing Behavior

The Hermes Langfuse plugin supports several optional environment variables:

Set these in ~/.hermes/.env or export them in your shell before starting Hermes.

Step 5: View Traces in Langfuse

After running the example, open Langfuse Cloud to see the full trace including prompts, completions, tool calls, token usage, and latency.