# traceloop

## Docs

- [Create an Auto Monitor Setup](https://www.traceloop.com/docs/api-reference/auto-monitor-setups/create-an-auto-monitor-setup.md): Trigger a process that attempts to create a Traceloop monitor using the provided selector criteria
- [Delete an auto monitor setup by external ID](https://www.traceloop.com/docs/api-reference/auto-monitor-setups/delete-an-auto-monitor-setup-by-external-id.md): Delete an auto monitor setup by its external ID
- [Get an auto monitor setup by external ID](https://www.traceloop.com/docs/api-reference/auto-monitor-setups/get-an-auto-monitor-setup-by-external-id.md): Get a specific auto monitor setup by its external ID
- [List auto monitor setups](https://www.traceloop.com/docs/api-reference/auto-monitor-setups/list-auto-monitor-setups.md): List all auto monitor setups for the organization with optional filters
- [Update an auto monitor setup by external ID](https://www.traceloop.com/docs/api-reference/auto-monitor-setups/update-an-auto-monitor-setup-by-external-id.md): Update an existing auto monitor setup by its external ID
- [Get costs by property](https://www.traceloop.com/docs/api-reference/costs/property_costs.md)
- [Create a custom LLM-as-a-judge evaluator](https://www.traceloop.com/docs/api-reference/evaluators/create-a-custom-llm-as-a-judge-evaluator.md): Create a new custom evaluator at the organization level without project or environment bindings. The evaluator uses LLM-as-a-judge to evaluate inputs based on the provided configuration. The response format is automatically generated from the output schema.
- [Execute agent-efficiency evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-agent-efficiency-evaluator.md): Evaluate agent efficiency - detect redundant calls, unnecessary follow-ups
- [Execute agent-flow-quality evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-agent-flow-quality-evaluator.md): Validate agent trajectory against user-defined conditions
- [Execute agent-goal-accuracy evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-agent-goal-accuracy-evaluator.md): Evaluate agent goal accuracy
- [Execute agent-goal-completeness evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-agent-goal-completeness-evaluator.md): Measure if agent accomplished all user goals
- [Execute agent-tool-error-detector evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-agent-tool-error-detector-evaluator.md): Detect errors or failures during tool execution
- [Execute agent-tool-trajectory evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-agent-tool-trajectory-evaluator.md): Compare actual tool calls against expected reference tool calls
- [Execute answer-completeness evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-answer-completeness-evaluator.md): Evaluate whether the answer is complete and contains all the necessary information
- [Execute answer-correctness evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-answer-correctness-evaluator.md): Evaluate factual accuracy by comparing answers against ground truth
- [Execute answer-relevancy evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-answer-relevancy-evaluator.md): Check if an answer is relevant to a question
- [Execute char-count evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-char-count-evaluator.md): Count the number of characters in text
- [Execute char-count-ratio evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-char-count-ratio-evaluator.md): Calculate the ratio of characters between two texts
- [Execute context-relevance evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-context-relevance-evaluator.md): Evaluate whether retrieved context contains sufficient information to answer the query
- [Execute conversation-quality evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-conversation-quality-evaluator.md): Evaluate conversation quality based on tone, clarity, flow, responsiveness, and transparency
- [Execute faithfulness evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-faithfulness-evaluator.md): Check if a completion is faithful to the provided context
- [Execute html-comparison evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-html-comparison-evaluator.md): Compare two HTML documents for structural and content similarity
- [Execute instruction-adherence evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-instruction-adherence-evaluator.md): Evaluate how well responses follow given instructions
- [Execute intent-change evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-intent-change-evaluator.md): Detect changes in user intent between prompts and completions
- [Execute json-validator evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-json-validator-evaluator.md): Validate JSON syntax
- [Execute perplexity evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-perplexity-evaluator.md): Measure text perplexity from logprobs
- [Execute pii-detector evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-pii-detector-evaluator.md): Detect personally identifiable information in text
- [Execute placeholder-regex evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-placeholder-regex-evaluator.md): Validate text against a placeholder regex pattern
- [Execute profanity-detector evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-profanity-detector-evaluator.md): Detect profanity in text
- [Execute prompt-injection evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-prompt-injection-evaluator.md): Detect prompt injection attempts
- [Execute prompt-perplexity evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-prompt-perplexity-evaluator.md): Measure prompt perplexity to detect potential injection attempts
- [Execute regex-validator evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-regex-validator-evaluator.md): Validate text against a regex pattern
- [Execute secrets-detector evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-secrets-detector-evaluator.md): Detect secrets and credentials in text
- [Execute semantic-similarity evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-semantic-similarity-evaluator.md): Calculate semantic similarity between completion and reference
- [Execute sexism-detector evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-sexism-detector-evaluator.md): Detect sexist language and bias
- [Execute sql-validator evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-sql-validator-evaluator.md): Validate SQL query syntax
- [Execute tone-detection evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-tone-detection-evaluator.md): Detect the tone of the text
- [Execute topic-adherence evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-topic-adherence-evaluator.md): Evaluate topic adherence
- [Execute toxicity-detector evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-toxicity-detector-evaluator.md): Detect toxic or harmful language
- [Execute uncertainty-detector evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-uncertainty-detector-evaluator.md): Detect uncertainty in the text
- [Execute word-count evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-word-count-evaluator.md): Count the number of words in text
- [Execute word-count-ratio evaluator](https://www.traceloop.com/docs/api-reference/evaluators/execute-word-count-ratio-evaluator.md): Calculate the ratio of words between two texts
- [Introduction](https://www.traceloop.com/docs/api-reference/introduction.md)
- [Get metrics high water mark](https://www.traceloop.com/docs/api-reference/metrics/get-metrics-high-water-mark.md): Returns the timestamp of the last successfully processed evaluation (high water mark)
- [Get metrics with filtering and grouping](https://www.traceloop.com/docs/api-reference/metrics/get-metrics-with-filtering-and-grouping.md): Retrieves metrics data with support for filtering, sorting, and pagination. Metrics are grouped by metric name with individual data points. Supports filtering by direct column fields (bool_value, trace_id, etc.), label fields (labels.agent_name, labels.trace_id), and attribute fields (attributes.*).
- [Create a new organization](https://www.traceloop.com/docs/api-reference/organizations/create-a-new-organization.md): Create a new organization with environments and API keys.
- [Delete specific user data](https://www.traceloop.com/docs/api-reference/privacy/delete_request.md)
- [Status of user deletion request](https://www.traceloop.com/docs/api-reference/privacy/delete_status.md)
- [Disable logging of prompts and responses for specific users](https://www.traceloop.com/docs/api-reference/tracing/delete_whitelisted_user.md)
- [Get identifiers of users that are allowed to be logged](https://www.traceloop.com/docs/api-reference/tracing/get_whitelisted_users.md)
- [Enable logging of prompts and responses](https://www.traceloop.com/docs/api-reference/tracing/whitelist_user.md)
- [Get Spans](https://www.traceloop.com/docs/api-reference/warehouse/get_spans.md)
- [Quick Start](https://www.traceloop.com/docs/datasets/quick-start.md)
- [SDK usage](https://www.traceloop.com/docs/datasets/sdk-usage.md): Access your managed datasets with the Traceloop SDK
- [Custom Evaluators](https://www.traceloop.com/docs/evaluators/custom-evaluator.md): Define an evaluator for your specific needs
- [Evaluator Slugs](https://www.traceloop.com/docs/evaluators/evaluator-slugs.md): Reference list of all built-in evaluator slugs for use with the API
- [Guardrails](https://www.traceloop.com/docs/evaluators/guardrails.md): Real-time evaluation and safety checks for LLM applications
- [Introduction](https://www.traceloop.com/docs/evaluators/intro.md): Evaluating workflows and LLM outputs
- [Made by Traceloop](https://www.traceloop.com/docs/evaluators/made-by-traceloop.md): Pre-configured evaluators by Traceloop for common assessment tasks
- [Introduction](https://www.traceloop.com/docs/experiments/introduction.md)
- [Result Overview](https://www.traceloop.com/docs/experiments/result-overview.md)
- [Run via SDK](https://www.traceloop.com/docs/experiments/running-from-code.md): Learn how to run experiments programmatically using the Traceloop SDK
- [Hub Configuration](https://www.traceloop.com/docs/hub/configuration.md): How to configure Traceloop Hub and connect it to different LLM providers
- [Getting Started with Traceloop Hub](https://www.traceloop.com/docs/hub/getting-started.md): Set up Hub as a smart proxy to all your LLM calls.
- [GitHub](https://www.traceloop.com/docs/integrations/github.md): Run experiments in CI and get evaluation results directly in your pull requests
- [Posthog](https://www.traceloop.com/docs/integrations/posthog.md): Connect Traceloop to Posthog to combine LLM insights with your product analytics
- [Slack](https://www.traceloop.com/docs/integrations/slack.md): Get daily or weekly messages about your AI flows directly in Slack
- [Introduction](https://www.traceloop.com/docs/introduction.md): Monitor, debug and test the quality of your LLM outputs
- [Defining Monitors](https://www.traceloop.com/docs/monitoring/defining-monitors.md): Learn how to create and configure monitors to evaluate your LLM outputs
- [Introduction](https://www.traceloop.com/docs/monitoring/introduction.md): Detect hallucinations and regressions in the quality of your LLMs
- [Using Monitors](https://www.traceloop.com/docs/monitoring/using-monitors.md): Learn how to view, analyze, and act on monitor results in your LLM applications
- [SDK Initialization Options](https://www.traceloop.com/docs/openllmetry/configuration.md): Documentation of the initialization options for the SDKs.
- [Local Development](https://www.traceloop.com/docs/openllmetry/contributing/developing.md)
- [Overview](https://www.traceloop.com/docs/openllmetry/contributing/overview.md): We welcome any contributions to  OpenLLMetry, big or small.
- [GenAI Semantic Conventions](https://www.traceloop.com/docs/openllmetry/contributing/semantic-conventions.md)
- [Go](https://www.traceloop.com/docs/openllmetry/getting-started-go.md): Install OpenLLMetry for Go by following these 3 easy steps and get instant monitoring.
- [Next.js](https://www.traceloop.com/docs/openllmetry/getting-started-nextjs.md): Install OpenLLMetry for Next.js by following these 3 easy steps and get instant monitoring.
- [Python](https://www.traceloop.com/docs/openllmetry/getting-started-python.md): Install OpenLLMetry for Python by following these 3 easy steps and get instant monitoring.
- [Ruby](https://www.traceloop.com/docs/openllmetry/getting-started-ruby.md): Install OpenLLMetry for Ruby by following these 3 easy steps and get instant monitoring.
- [Node.js](https://www.traceloop.com/docs/openllmetry/getting-started-ts.md): Install OpenLLMetry for Node.js by following these 3 easy steps and get instant monitoring.
- [LLM Observability with Axiom and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/axiom.md)
- [Azure Application Insights](https://www.traceloop.com/docs/openllmetry/integrations/azure.md)
- [LLM Observability with BMC and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/bmc.md)
- [LLM Observability with Braintrust and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/braintrust.md)
- [LLM Observability with Dash0 and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/dash0.md)
- [LLM Observability with Datadog and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/datadog.md)
- [LLM Observability with Dynatrace and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/dynatrace.md)
- [LLM Observability with Elasticsearch APM Service](https://www.traceloop.com/docs/openllmetry/integrations/elasticsearch-apm.md)
- [LLM Observability with Google Cloud and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/gcp.md)
- [LLM Observability with Grafana and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/grafana.md)
- [LLM Observability with groundcover and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/groundcover.md)
- [LLM Observability with Highlight and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/highlight.md)
- [LLM Observability with Honeycomb and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/honeycomb.md)
- [LLM Observability with HyperDX and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/hyperdx.md)
- [LLM Observability with Instana and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/instana.md)
- [Overview](https://www.traceloop.com/docs/openllmetry/integrations/introduction.md): Connect to any observability platform - Traceloop, Dynatrace, Datadog, Honeycomb, and others
- [LLM Observability with KloudMate and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/kloudmate.md)
- [LLM observability with Laminar and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/laminar.md)
- [Enhance LLM Observability with Langfuse and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/langfuse.md)
- [LLM Observability with LangSmith and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/langsmith.md)
- [LLM Observability with Middleware and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/middleware.md)
- [LLM observability with New Relic and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/newrelic.md)
- [LLM Observability with Oracle Cloud Infrastructure Application Performance Monitoring(APM) service](https://www.traceloop.com/docs/openllmetry/integrations/oraclecloud.md)
- [LLM observability with OpenTelemetry Collector](https://www.traceloop.com/docs/openllmetry/integrations/otel-collector.md)
- [LLM Observability with Scorecard and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/scorecard.md)
- [LLM Observability with Sentry and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/sentry.md)
- [LLM Observability with Service Now Cloud Observability and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/service-now.md)
- [LLM Observability with SigNoz and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/signoz.md)
- [LLM Observability with Splunk and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/splunk.md)
- [LLM Observability with Tencent APM and OpenLLMetry](https://www.traceloop.com/docs/openllmetry/integrations/tencent.md)
- [LLM Observability with Traceloop](https://www.traceloop.com/docs/openllmetry/integrations/traceloop.md)
- [What is OpenLLMetry?](https://www.traceloop.com/docs/openllmetry/introduction.md)
- [Telemetry](https://www.traceloop.com/docs/openllmetry/privacy/telemetry.md)
- [Prompts, Completions and Embeddings](https://www.traceloop.com/docs/openllmetry/privacy/traces.md)
- [Workflow Annotations](https://www.traceloop.com/docs/openllmetry/tracing/annotations.md): Enrich your traces by annotating chains and workflows in your app
- [Associating Entities with Traces](https://www.traceloop.com/docs/openllmetry/tracing/association.md): How to associate traces with entities in your own application
- [Issues with Auto-instrumentation (Typescript / Javascript)](https://www.traceloop.com/docs/openllmetry/tracing/js-force-instrumentations.md): How to overcome issues with automatic instrumentations on Next.js and some Webpack environments.
- [Manually reporting calls to LLMs and Vector DBs](https://www.traceloop.com/docs/openllmetry/tracing/manual-reporting.md): What should I do if my favorite vector DB or LLM is not supported by OpenLLMetry?
- [Multi-Modality Support](https://www.traceloop.com/docs/openllmetry/tracing/multi-modality.md): Automatic logging and visualization of multi-modal LLM interactions
- [Usage with Threads (Python)](https://www.traceloop.com/docs/openllmetry/tracing/python-threads.md): How to use OpenLLMetry with `ThreadPoolExecutor` and other thread-based libraries.
- [Sessions](https://www.traceloop.com/docs/openllmetry/tracing/sessions.md): Group traces from multi-turn conversations into sessions
- [What's Supported?](https://www.traceloop.com/docs/openllmetry/tracing/supported.md): A list of the models, vector DBs and frameworks that OpenLLMetry supports out of the box.
- [Tracking User Feedback](https://www.traceloop.com/docs/openllmetry/tracing/user-feedback.md)
- [Versioning](https://www.traceloop.com/docs/openllmetry/tracing/versions.md): Learn how to enrich your traces by versioning your workflows and prompts
- [Without OpenLLMetry SDK](https://www.traceloop.com/docs/openllmetry/tracing/without-sdk.md)
- [Troubleshooting](https://www.traceloop.com/docs/openllmetry/troubleshooting.md): Not seeing anything? Here are some things to check.
- [Column Management](https://www.traceloop.com/docs/playgrounds/columns/column-management.md): Learn all columns general functionalities
- [Data Columns](https://www.traceloop.com/docs/playgrounds/columns/data-columns.md)
- [Prompt Column](https://www.traceloop.com/docs/playgrounds/columns/prompt.md): Execute LLM prompts with full model configuration
- [Quick Start](https://www.traceloop.com/docs/playgrounds/quick-start.md)
- [Quick Start](https://www.traceloop.com/docs/prompts/quick-start.md)
- [Prompt Registry](https://www.traceloop.com/docs/prompts/registry.md): Manage your prompts on the Traceloop platform
- [Fetching Prompts](https://www.traceloop.com/docs/prompts/sdk-usage.md): Use your managed prompts with the Traceloop SDKs
- [Full Platform Self-Hosting](https://www.traceloop.com/docs/self-host/full-deployment.md): Deploy the complete Traceloop platform in your infrastructure
- [Hybrid Deployment](https://www.traceloop.com/docs/self-host/hybrid-deployment.md): Set up Traceloop with data sovereignty
- [Self-Hosting Overview](https://www.traceloop.com/docs/self-host/introduction.md): Learn about Traceloop's deployment options
- [Troubleshooting Guide](https://www.traceloop.com/docs/self-host/troubleshooting.md): Solutions to common issues when self-hosting Traceloop
- [Managing API Keys](https://www.traceloop.com/docs/settings/managing-api-keys.md): Generate and manage API keys for sending traces and accessing Traceloop features
- [Projects and Environments](https://www.traceloop.com/docs/settings/projects-and-environments.md): Organize your applications and deployment stages with Projects and Environments

## OpenAPI Specs

- [openapi](https://www.traceloop.com/docs/openapi.json)

## Optional

- [Community](https://traceloop.com/slack)
- [GitHub](https://github.com/traceloop)