If you’re on Next.js, follow the Next.js guide.

1

Install the SDK

Run the following command in your terminal:

npm install @traceloop/node-server-sdk

In your LLM app, initialize the Traceloop tracer like this:

import * as traceloop from "@traceloop/node-server-sdk";

traceloop.initialize();

Because of the way Javascript works, you must import the Traceloop SDK before importing any LLM module like OpenAI.

If you’re running this locally, you may want to disable batch sending, so you can see the traces immediately:

traceloop.initialize({ disableBatch: true });
2

Annotate your workflows

If you have complex workflows or chains, you can annotate them to get a better understanding of what’s going on. You’ll see the complete trace of your workflow on Traceloop or any other dashboard you’re using.

We have a set of methods and decorators to make this easier. Assume you have a function that renders a prompt and calls an LLM, simply wrap it in a withWorkflow() function call.

We also have compatible Typescript decorators for class methods which are more convenient.

If you’re using an LLM framework like Haystack, Langchain or LlamaIndex - we’ll do that for you. No need to add any annotations to your code.

async function suggestAnswers(question: string) {
  return await withWorkflow({ name: "suggestAnswers" }, () => {
    ...
  });
}

For more information, see the dedicated section in the docs.

3

Configure trace exporting

Lastly, you’ll need to configure where to export your traces. The 2 environment variables controlling this are TRACELOOP_API_KEY and TRACELOOP_BASE_URL.

For Traceloop, read on. For other options, see Exporting.

Using Traceloop Cloud

Go to Traceloop, and create a new account. Then, click on Environments on the left-hand navigation bar. Or go to directly to https://app.traceloop.com/settings/api-keys. Click Generate API Key to generate an API key for the development environment and click Copy API Key to copy it over.

Make sure to copy it as it won’t be shown again.

Set the copied Traceloop’s API key as an environment variable in your app named TRACELOOP_API_KEY.

Done! You’ll get instant visibility into everything that’s happening with your LLM. If you’re calling a vector DB, or any other external service or database, you’ll also see it in the Traceloop dashboard.

Was this page helpful?

PythonNext.js