Observability for Bedrock with Dynatrace

Step 1

Install the Traceloop SDK and initialize it. It will automatically log all calls to Bedrock using boto3 - both prompts and completions.

import boto3
import json

from traceloop.sdk import Traceloop

Traceloop.init(app_name="joke_generation_service")
brt = boto3.client(service_name='bedrock-runtime')

body = json.dumps({
    "prompt": "Tell me a joke about opentelemetry",
    "max_tokens": 200,
    "temperature": 0.5,
    "p": 0.5,
})

response = brt.invoke_model(
    body=body,
    modelId='cohere.command-text-v14',
    accept='application/json',
    contentType='application/json'
)

response_body = json.loads(response.get('body').read())

response = response_body.get('generations')[0].get('text')
print(response)

import * as traceloop from "@traceloop/node-server-sdk";
import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from "@aws-sdk/client-bedrock-runtime";

traceloop.initialize({
  appName: "sample_bedrock_ai21",
  apiKey: process.env.TRACELOOP_API_KEY,
  disableBatch: true,
});

// Create a BedrockRuntimeClient with your configuration
const client = new BedrockRuntimeClient();

const prompt = "How do you say Good Morning in French?";

const input = {
  modelId: "ai21.j2-ultra-v1",
  contentType: "application/json",
  accept: "application/json",
  body: JSON.stringify({
    prompt,
    maxTokens: 200,
    temperature: 0.7,
    topP: 1,
    stopSequences: [],
    presencePenalty: { scale: 0 },
    frequencyPenalty: { scale: 0 },
  }),
};

async function generateTextContent() {
  return await traceloop.withWorkflow("sample_completion", {}, async () => {
    // Create an InvokeModelCommand with the input parameters
    const command = new InvokeModelCommand(input);

    // Send the command to invoke the model and await the response
    client.send(command).then((response) => {
      // Save the raw response
      const rawRes = response.body;

      // Convert it to a JSON String
      const jsonString = new TextDecoder().decode(rawRes);

      // Parse the JSON string
      const parsedResponse = JSON.parse(jsonString);

      console.log(">>> non-stream", parsedResponse);
    });
  });
}

generateTextContent();

Step 2

Go to your Dynatrace environment and create a new access token under Manage Access Tokens. The access token needs the following permission scopes that allow the ingest of OpenTelemetry spans, metrics and logs (openTelemetryTrace.ingest, metrics.ingest, logs.ingest). Set the following environment variables.

TRACELOOP_BASE_URL=https://<YOUR_ENV>.live.dynatrace.com\api\v2\otlp
TRACELOOP_HEADERS=Authorization=Api-Token%20<YOUR_ACCESS_TOKEN>

Docs

Observability for Bedrock with Dynatrace

Step 1

Step 2

Discover use cases

Trace prompts and completions in Bedrock

More things you can connect to {platform}

Chroma

Cohere

OpenAI

Watsonx

Anthropic

LangChain

LlamaIndex