Start calling this model with endpoint-specific examples.
Headline benchmark standings and comparison context.
Key dates, capabilities, and model metadata.
Start calling this model with endpoint-specific examples.
Headline benchmark standings and comparison context.
Key dates, capabilities, and model metadata.
Start calling this model with endpoint-specific examples.
Headline benchmark standings and comparison context.
Key dates, capabilities, and model metadata.
18 Apr 2024
Input
Output
Headline benchmark standings and comparison context.
Top benchmark results for meta/llama-3-8b.
Detailed benchmark comparisons now live in the Compare tool.
Start calling this model with endpoint-specific examples.
Choose a supported endpoint, pick a main language, then select the example style you want to copy.
import AIStats from '@ai-stats/sdk';
const client = new AIStats({
apiKey: process.env.AI_STATS_API_KEY,
});
const response = await client.generateResponse({
"model": "meta/llama-3-8b",
"input": "Give me one fun fact about cURL.",
"service_tier": "standard"
});
const outputText = response.output
?.flatMap((item) => item.content ?? [])
.find((item) => item.type === "output_text")
?.text;
console.log(outputText ?? response);Parameters
Aggregated across active providers for the responses route.
Routing will select a compatible provider when a parameter narrows availability, so this list stays model-facing instead of provider-facing.
| Parameter | Description |
|---|---|
temperature | Controls how random token selection can be. |
top_p | Applies nucleus sampling by limiting candidates to a probability mass threshold. |
top_k | Restricts sampling to the top-k candidate tokens on providers that expose it. |
max_tokens | Caps output length on endpoints and providers that use the max_tokens field name. |
frequency_penalty | Discourages repeated tokens in proportion to how often they already appeared. |
presence_penalty | Encourages the model to explore new wording or topics after they first appear. |
repetition_penalty | Applies provider-specific anti-repetition behavior outside the classic penalty fields. |
seed | Requests deterministic sampling when the upstream provider supports seeded generation. |
stop | Defines one or more sequences that terminate generation early. |
logprobs | Requests token-level probability data in the response. |
logit_bias | Adjusts token selection bias directly when a provider exposes logit control. |
min_p | Narrows sampling by discarding tokens below a minimum probability threshold. |
top_logprobs | Limits how many alternative token probabilities are returned per position. |
Select a route to update the request snippet and compatibility details.
/v1/responses/v1/chat/completions/v1/messagesCore latency and throughput trends from recent traffic.
Latency
Throughput
Uptime
Latency
Throughput
Uptime
Latency
Throughput
Uptime
Latency
Throughput
Uptime
