This guide provides an overview of the core concepts and terminology specific to model routing in Not Diamond.

Model routing

Model routing is the process of intelligently selecting the best LLM for each query from a set of candidate models. Not Diamond's router analyzes your input and predicts which model will provide the optimal response based on quality, cost, or latency tradeoffs. Instead of using a single model for all queries, routing leverages the strengths of different models to maximize performance while minimizing costs.

Learn more: What is Model Routing? | Quick Start

Session ID

A session ID is a unique identifier returned by Not Diamond when you make a routing request. This ID allows you to:

• Track which model was selected for a specific query
• Debug and analyze routing behavior across your application

Example:

result = client.model_router.select_model(
    messages=[{"role": "user", "content": "Hello"}],
    llm_providers=[{"provider": "openai", "model": "gpt-5-2025-08-07"}]
)

print(result.session_id)  # e.g., "550e8400-e29b-41d4-a716-446655440000"

Preference ID

A preference ID is an identifier for a specific custom router.

Tradeoffs

Tradeoffs allow you to balance model selection based on cost, latency, or quality. Not Diamond uses Pareto optimization to:

• Cost tradeoff - Prefer cheaper models when quality is comparable
• Latency tradeoff - Favor faster models to reduce response time
• Quality tradeoff (default) - Prioritize the highest quality responses

You can dynamically adjust these tradeoffs based on your application's needs at request time.

# Optimize for cost
result = client.model_router.select_model(
    messages=[{"role": "user", "content": "Hello"}],
    llm_providers=[
        {"provider": "openai", "model": "gpt-5-2025-08-07"},
        {"provider": "openai", "model": "gpt-5-mini-2025-08-07"}
    ],
    tradeoff="cost"  # Options: "cost", "latency", or None (quality)
)

Pre-trained vs. custom routers

Not Diamond offers two types of routers:

• Pre-trained router - A general-purpose router trained on cross-domain data, ready to use out of the box
• Custom router - A router trained on your specific evaluation data, optimized for your use case and quality criteria

Pre-trained routers are ideal for getting started quickly or for general-purpose applications. Custom routers are recommended for production applications where domain-specific performance is critical.

Learn more: Training a Custom Router