Documentation

Model gateway

Suggest Edits

If you're already using an OpenAI compatible LLM endpoint, you can onboard Not Diamond easily by using our model gateway. Simply change the base URL, add your provider API keys to the dashboard, and you can begin dynamically routing your queries to the most suitable LLM on every call.

1. Add your API keys

Head to the "AI Providers" page at app.notdiamond.ai and insert your API keys. The gateway will automatically retrieve the relevant provider key depending on the model recommended.

📘

Your keys are encrypted using AES256

Not Diamond uses the highest level of security (AES256) to encrypt your API keys in the database.

2. Change the base URL

The base URL of the gateway is https://proxy.notdiamond.ai/v1/proxy.

from openai import OpenAI
import os

client = OpenAI(
  base_url="https://proxy.notdiamond.ai/v1/proxy",
  api_key=os.environ["NOTDIAMOND_API_KEY"],
)

response = client.chat.completions.create(
  model="notdiamond", # this is so the API body checker doesn't complain
  extra_body={
    "models": ["gpt-4o", "claude-3-5-sonnet-20240620"], # pass in the LLM options you want to route between
    "tradeoff": "cost",
    "preference_id": "YOUR_PREFERENCE_ID"
  },
	messages=[{"role": "user", "content": "What is a proxy?"}]
)
print(response.choices[0].message.content)

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://proxy.notdiamond.ai/v1/proxy",
  apiKey: process.env.NOTDIAMOND_API_KEY,
});

async function main() {
  const response = await client.chat.completions.create({
    models: ["gpt-4o", "claude-3-5-sonnet-20240620"],
    messages: [{ role: "user", content: "What is a proxy?" }],
    tradeoff: "cost",
    preference_id: "YOUR_PREFERENCE_ID"
  });
  console.log(response.choices[0].message.content);
}

main();

curl https://proxy.notdiamond.ai/v1/proxy/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "models": ["gpt-4o", "claude-3-5-sonnet-20240620"],
    "messages": [
      {
        "role": "user",
        "content": "What is a proxy?"
      }
    ],
    "tradeoff": "cost",
    "preference_id": "YOUR_PREFERENCE_ID"
  }' \
  -H "Authorization: Bearer $NOTDIAMOND_API_KEY"

Updated about 1 month ago