Integration: Traceloop

Evaluate and monitor the quality of your LLM apps and agents

Authors

Traceloop Team

• OpenLLMetry
  • Installation
  • Example
  • About Traceloop

OpenLLMetry

OpenLLMetry is an open-source Python package built and maintained by Traceloop that instruments your Haystack-based applications with OpenTelemetry. This gives you full visibility to your LLM app, right in your existing observability stack. You can also connect this to Traceloop to get quality evaluation metrics and LLM-specific capabilities like Prompt Playground.

More info on the documentation.

Installation

pip install traceloop-sdk

Example

Basic integration is as simple as adding one line to your code:

Traceloop.init(app_name="your_app_name")

When you run your code, you’ll get a link to the Traceloop UI where you can see your app’s traces. If you want to connect to a different observability platform, follow the guide for exporting your traces.

Use a Traceloop API Key

If you have an account with Traceloop and would like to see your traces on your account dashboard:

• Create an API key on Traceloop
• Export the API key in an environment variable called TRACELOOP_API_KEY

Trace Haystack Pipelines

Once you’ve initialized a Traceloop app, any Haystack pipeline that you run in the same environment will get logged in the dashboard provided by the generated Traceloop URL. For example, below is a simple Haystack pipeline and its traceloop logs:

from haystack.nodes import PromptNode, PromptTemplate, AnswerParser
from haystack.pipelines import Pipeline
from traceloop.sdk import Traceloop

Traceloop.init(app_name="haystack_app")

prompt = PromptTemplate(
    prompt="Tell me a joke about {query}\n",
    output_parser=AnswerParser(),
)

prompt_node = PromptNode(
    model_name_or_path="gpt-4",
    api_key=api_key,
    default_prompt_template=prompt,
)

pipeline = Pipeline()
pipeline.add_node(component=prompt_node, name="PromptNode", inputs=["Query"])
result = pipeline.run("Haystack")

About Traceloop

Traceloop is a platform for monitoring, evaluating and debugging LLM apps and agents. Deploy changes with confidence and get insights into your LLM executions.

Key features

• Manage your prompts in a single place with version support, gradual rollout, A/B testing, and more.
• Evaluate your prompts and models quality with auto-generated test sets.
• Monitor your LLM app’s performance and get alerts when it’s not behaving as expected.