Overview

The Inference class is the main facade for interacting with LLM APIs. It provides a clean, immutable interface for chat completions, tool calling, JSON output generation, and streaming — all through a consistent API regardless of the underlying provider.

Quick Start

The simplest way to generate text is with a single chained call:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$answer = Inference::using('openai')
    ->withMessages(Messages::fromString('What is the capital of France?'))
    ->get();
// @doctest id="9462"

The using() static method resolves a named preset from your configuration, while withMessages() accepts a Messages object. Use Messages::fromString() to wrap a plain text prompt, or Messages::fromArray() to convert an array of role/content pairs. The get() method executes the request and returns the complete assistant Message.

Creating an Inference Instance

For more control over the lifecycle, create an instance directly. Without arguments, Inference uses a sensible default configuration:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$inference = new Inference();

$answer = $inference
    ->withMessages(Messages::fromArray([['role' => 'user', 'content' => 'Explain event sourcing briefly.']]))
    ->get();
// @doctest id="0fc4"

You may also use the with() method, which accepts all request parameters at once:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$answer = (new Inference)->with(
    messages: Messages::fromString('What is the capital of France?'),
)->get();
// @doctest id="72ed"

Core Request Fields

The with() method and its individual with...() counterparts allow you to set every aspect of the inference request:

Field	Method	Description
messages	withMessages()	The conversation messages
model	withModel()	Override the model defined in the preset
tools	withTools()	Tool/function definitions (ToolDefinitions) for the model to call
toolChoice	withToolChoice()	Control which tool the model should use (ToolChoice)
responseFormat	withResponseFormat()	Request structured output (ResponseFormat)
options	withOptions()	Provider-specific parameters (temperature, max_tokens, etc.)
maxTokens	withMaxTokens()	Shorthand for setting the maximum output token count

Execution Paths

Once you have configured a request, choose how to execute it:

Method	Returns	Use case
get()	Message	Complete assistant turn
response()	InferenceResponse	Assistant Message with usage, finish reason, and provider data
asJson()	string	Extract JSON from the response content
asJsonData()	array	Decode JSON from the response into a PHP array
asToolCallJson()	string	Extract tool call arguments as a JSON string
asToolCallJsonData()	array	Decode tool call arguments into a PHP array
stream()	InferenceStream	Stream partial deltas as they arrive

Multi-Turn Conversations

For multi-turn conversations, pass an array of messages with role annotations:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$messages = Messages::fromArray([
    ['role' => 'user', 'content' => 'Can you help me with a math problem?'],
    ['role' => 'assistant', 'content' => 'Of course! What would you like to solve?'],
    ['role' => 'user', 'content' => 'What is the square root of 144?'],
]);

$answer = Inference::using('openai')
    ->withMessages($messages)
    ->get();
// @doctest id="3633"

Customizing Request Options

Provider-specific parameters such as temperature, max_tokens, or top_p are passed through the options array. Most providers follow the OpenAI-compatible parameter conventions:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$answer = Inference::using('openai')
    ->withMessages(Messages::fromString('Write a short poem about coding.'))
    ->withModel('gpt-4o')
    ->withOptions(['temperature' => 0.7, 'max_tokens' => 200])
    ->get();
// @doctest id="3ac7"

You can also set all parameters at once via the with() convenience method:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$answer = Inference::using('openai')->with(
    messages: Messages::fromString('Write a haiku about PHP.'),
    model: 'gpt-4o',
    options: ['temperature' => 0.9, 'max_tokens' => 100],
)->get();
// @doctest id="4f3c"

Streaming Responses

Streaming lets you display partial output as it arrives from the model, creating a more responsive user experience. Call stream() to get an InferenceStream, then iterate over deltas:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$stream = Inference::using('openai')
    ->withMessages(Messages::fromString('Describe the capital of Brasil.'))
    ->withMaxTokens(512)
    ->stream();

foreach ($stream->deltas() as $delta) {
    echo $delta->messageChunks->textDelta();
}
// @doctest id="9530"

Each PartialInferenceDelta exposes ordered messageChunks; textDelta() projects the incremental text fragments. The stream also provides functional-style helpers — map(), filter(), and reduce() — for processing deltas inline.

You can also register a callback to handle each delta as it arrives:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$stream = Inference::using('openai')
    ->withMessages(Messages::fromString('Tell me a story.'))
    ->stream();

$stream->onDelta(fn($delta) => print($delta->messageChunks->textDelta()));

// Drain the stream to trigger callbacks
$stream->all();
// @doctest id="a0ce"

After the stream completes, call final() to retrieve the assembled InferenceResponse with full content and usage statistics.

Working with the Full Response

When you need more than just text, use response() to access the complete InferenceResponse object:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$response = Inference::using('openai')
    ->withMessages(Messages::fromString('What is quantum computing?'))
    ->response();

$message = $response->message();
$text = $message->content()->toString();
$usage = $response->usage();
$finishReason = $response->finishReason();
// @doctest id="fe02"

The response envelope carries the complete assistant message, token usage statistics, finish reason, and raw HTTP response data. Text, reasoning, and tool calls are derived from the message’s ordered parts.

Switching Between Providers

Polyglot ships with YAML-based presets for many providers. Switching between them is a single method call:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$question = Messages::fromString('What is the capital of France?');

$openai = Inference::using('openai')->withMessages($question)->get();
$anthropic = Inference::using('anthropic')->withMessages($question)->get();
$gemini = Inference::using('gemini')->withMessages($question)->get();
// @doctest id="2361"

Available presets include openai, anthropic, gemini, mistral, groq, ollama, fireworks, together, openrouter, cohere, deepseek, xai, azure, perplexity, sambanova, and others. Each preset is defined in a YAML file under resources/config/llm/presets/.

Configuring Presets

Each preset is a YAML file that defines the connection parameters for a provider. For example, the OpenAI preset:

driver: openai
apiUrl: 'https://api.openai.com/v1'
apiKey: '${OPENAI_API_KEY}'
endpoint: /chat/completions
model: gpt-4.1-nano
maxTokens: 1024
# @doctest id="db63"

Polyglot resolves presets from several locations, searched in order:

1. config/llm/presets/ (your project root)
2. packages/polyglot/resources/config/llm/presets/ (monorepo)
3. vendor/cognesy/instructor-php/packages/polyglot/resources/config/llm/presets/
4. vendor/cognesy/instructor-polyglot/resources/config/llm/presets/

To customize a provider, copy the relevant YAML file into config/llm/presets/ at your project root and modify it as needed. Environment variables are referenced with the ${VAR_NAME} syntax.

Selecting a Model

Each preset defines a default model, but you can override it per-request:

<?php
use Cognesy\Polyglot\Inference\Inference;
use Cognesy\Messages\Messages;

$answer = Inference::using('openai')
    ->withMessages(Messages::fromString('Explain machine learning in one sentence.'))
    ->withModel('gpt-4o')
    ->get();
// @doctest id="7538"

Immutability

Inference is immutable from the caller’s perspective. Every with...() method returns a new instance, leaving the original unchanged. This makes it safe to build a base configuration and derive specialized variants from it:

<?php
use Cognesy\Polyglot\Inference\Inference;

$base = Inference::using('openai')->withOptions(['temperature' => 0.3]);

$precise = $base->withModel('gpt-4o');
$fast = $base->withModel('gpt-4.1-mini');
// @doctest id="7bab"

Both $precise and $fast inherit the temperature setting without affecting each other or the $base instance.