Documentation

Personalized routing with feedback

Suggest Edits

Not Diamond can continuously personalize routing in real-time to application end-users based on their feedback. We'll run through a quick example to see how this works.

First, we'll create a new preference ID to represent our custom router. Then we'll take thumbs up and down signals to submit feedback and improve recommendations:

from notdiamond import NotDiamond, Metric

# Define your NotDiamond routing client
client = NotDiamond()

# Define the LLMs you'd like to route between
llm_providers = ['openai/gpt-3.5-turbo', 'openai/gpt-4-turbo-2024-04-09', 'openai/gpt-4o-2024-05-13',
                 'anthropic/claude-3-haiku-20240307', 'anthropic/claude-3-opus-20240229']

# instantiate an NDMetric object with accuracy as the feedback method
metric = Metric("accuracy")

# Call the client by passing in an array of messages, our selected models, and our preference_id
result, session_id, provider = client.chat.completions.create(
    messages=[
        {"role": "system", "content": "You are a world class programmer."},
        {"role": "user", "content": "Write a merge sort in Python."},
    ],
    model=llm_providers,
    preference_id="YOUR_PREFERENCE_ID",
    metric=metric
)

# Application logic...
# Let's assume the user submitted a thumbs up to the response
score = metric.feedback(session_id=session_id, llm_config=provider, value=1)

import {
    FeedbackSuccessResponse,
    NotDiamond,
  } from 'notdiamond';
  
  const notDiamond = new NotDiamond({
    apiKey: process.env.NOTDIAMOND_API_KEY,
  });
  
  async function main() {
    try {
      await provideFeedback('YOUR_PREFERENCE_ID');
    } catch (error) {
      console.error('An unexpected error occurred:', error);
    }
  }
  
  async function provideFeedback(
    sessionId: string,
  ): Promise<FeedbackSuccessResponse | null> {
    const result = await notDiamond.feedback({
      sessionId: 'SESSION_ID', // The session ID for the model select call you're giving feedback on
      feedback: { accuracy: 1 }, // Either 0 or 1
      provider: { provider: 'PROVIDER_CALLED', model: 'MODEL_CALLED' }, // E.g., 'openai' and 'gpt-4'
    });
  
    if ('detail' in result) {
      console.error(result.detail);
      return null;
    }
  
    const { session_id: feedbackSessionId } = result;
    console.log({ feedbackSessionId });
    return result;
  }
  
  void main();

🚧

Preference ID required

You must include a preference ID to submit feedback.

In this example, we submit feedback on the accuracy metric using one of two possible values:

  1. 0 if the response did not satisfy your requirement, or
  2. 1 if the response did satisfy your requirement.

As Not Diamond receives more and more feedback, it will continuously improve its routing recommendations to optimize for positive feedback.

👍

Try it out!

You can see what personalized routing feels like in our chat app.

Updated 3 months ago