Documentation

Quickstart

Suggest Edits

Not Diamond intelligently identifies which LLM is best-suited to respond to any given query. In this example we'll learn how Not Diamond works by making our first API request.

👍

Try it in Colab

You can follow along with the Python and TypeScript code below, or try it in Colab.

Installation

Python: Requires Python 3.10+. It’s recommended that you create and activate a virtualenv prior to installing the package. For this example, we'll be installing the optional additional create dependencies, which you can learn more about here.

pip install "notdiamond[create]"

npm install notdiamond dotenv

Setting up

Create a .env file with your Not Diamond API key and the API keys of the models you want to route between:

NOTDIAMOND_API_KEY = "YOUR_NOTDIAMOND_API_KEY"
OPENAI_API_KEY = "YOUR_OPENAI_API_KEY"
ANTHROPIC_API_KEY = "YOUR_ANTHROPIC_API_KEY"

You can also define API keys programmatically.

Sending your first Not Diamond API request

Create a new file in the same directory as your .env file and copy and run the code below (you can toggle between Python and TypeScript in the top left of the code block):

from notdiamond import NotDiamond

# Define the Not Diamond routing client
client = NotDiamond()

# The best LLM is determined by Not Diamond based on the messages and specified models
result, session_id, provider = client.chat.completions.create(
    messages=[ 
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Concisely explain merge sort."}  # Adjust as desired
    ],
    model=['openai/gpt-4o', 'openai/gpt-4o-mini', 'anthropic/claude-3-5-sonnet-20240620']
)

print("Not Diamond session ID: ", session_id)  # A unique ID of Not Diamond's recommendation
print("LLM called: ", provider.model)  # The LLM routed to
print("LLM output: ", result.content)  # The LLM response

import { NotDiamond } from 'notdiamond';
import dotenv from 'dotenv';
dotenv.config();

// Initialize the Not Diamond client
const notDiamond = new NotDiamond();

// The best LLM is determined by Not Diamond based on the messages and specified models
const result = await notDiamond.create({
  messages: [
    { role: 'system', content: 'You are a world class programmer.' },
    { role: 'user', content: 'Consiely explain merge sort.' }  // Adjust as desired
  ],
  llmProviders: [
    { provider: 'openai', model: 'gpt-4o' },
    { provider: 'openai', model: 'gpt-4o-mini' },
    { provider: 'anthropic', model: 'claude-3-5-sonnet-20240620' }
  ]
});

if ('detail' in result) {
  console.error('Error:', result.detail);
} 
else {
  console.log('Not Diamond session ID:', result.session_id);  // A unique ID of Not Diamond's recommendation
  console.log('LLM called:', result.providers);  // The LLM routed to
  console.log('LLM output', result.content); // The LLM response
}

Breaking down this example

We first define the routing client, which you can think of as a meta-LLM in which we'll combine multiple LLMs. We can define various clients, each with different configurations for different purposes, throughout our application.

client = NotDiamond()

// Initialize the Not Diamond client
const notDiamond = new NotDiamond();

After initializing the client and defining our LLMs, we next pass in an array of messages and the models we want to route between:

result, session_id, provider = client.chat.completions.create(
    messages=[ 
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Concisely explain merge sort."}  # Adjust as desired
    ],
    model=['openai/gpt-4o', 'openai/gpt-4o-mini', 'anthropic/claude-3-5-sonnet-20240620']
)

const result = await notDiamond.create({
  messages: [
    { role: 'system', content: 'You are a world class programmer.' },
    { role: 'user', content: 'Consiely explain merge sort.' }  // Adjust as desired
  ],
  llmProviders: [
    { provider: 'openai', model: 'gpt-4o' },
    { provider: 'openai', model: 'gpt-4o-mini' },
    { provider: 'anthropic', model: 'claude-3-5-sonnet-20240620' }
  ]
});

This returns a session ID and a recommended model:

  • Session ID: a unique ID for this specific recommendation. This is useful for submitting feedback on routing decisions.
  • Provider: the LLM selected by the ND API as the most appropriate for responding to the query.
  • LLM response (Python only): in addition to returning a recommended LLM, the Not Diamond Python SDK can also facilitate client-side requests to the recommended LLM with the create method. Alternatively, we can use model_select to simply return a session ID and a provider. You can learn more about these two methods here

📘

Good for use cases with diverse inputs

Not Diamond's out-of-the-box router (which we leverage in this example) is most useful for applications in which we are handling diverse inputs, such as a chatbot or a code generation assistant. For narrower tasks, we can train a custom router optimized to our data.

Next steps

In this example, we've learned how to dynamically route an array of messages to best-suited LLM amongst a set of various candidates. To explore all the features that Not Diamond offers, checkout the following guides

Updated about 2 months ago