Prompts, Completions and Embeddings
By default, OpenLLMetry logs prompts, completions, and embeddings to span attributes. This gives you a clear visibility into how your LLM application is working, and can make it easy to debug and evaluate the quality of the outputs.
However, you may want to disable this logging for privacy reasons, as they may contain highly sensitive data from your users. You may also simply want to reduce the size of your traces.
Disabling logging globally
To disable logging, set the TRACELOOP_TRACE_CONTENT
environment variable to false
.
On Typescript/Javascript you can also pass the traceContent
option.
OpenLLMetry SDK, as well as all individual instrumentations will respect this setting.
Enabling logging selectively in specific workflows / tasks
You can decide to selectively enable prompt logging for specific workflows, tasks, agents, or tools, using the annotations API.
If you don’t specify a traceContent
option, the global setting will be used.
Enabling logging selectively for specific users
You can decide to selectively enable or disable prompt logging for specific users or workflows.
Using the Traceloop Platform
We have an API to enable content tracing for specific users, as defined by association entities. See the Traceloop API documentation for more information.
Without the Traceloop Platform
Set a key called override_enable_content_tracing
in the OpenTelemetry context to True
right before making the LLM call
you want to trace with prompts.
This will create a new context that will instruct instrumentations to log prompts and completions as span attributes.
from opentelemetry.context import attach, set_value
attach(set_value("override_enable_content_tracing", True))
Make sure to also disable it afterwards:
from opentelemetry.context import attach, set_value
attach(set_value("override_enable_content_tracing", False))