> ## Documentation Index
> Fetch the complete documentation index at: https://docs.instructorphp.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Lifecycle

Understanding the request lifecycle helps when debugging provider issues, implementing custom drivers, or hooking into events for observability. This page traces the complete flow for both inference and embeddings operations.

## Inference Lifecycle

### 1. Request Construction

The lifecycle begins when the application builds an `InferenceRequest` through the `Inference` facade:

```php theme={null}
$inference = Inference::using('openai')
    ->withMessages(Messages::fromString('Explain PHP generics.'))
    ->withModel('gpt-4.1-nano')
    ->withMaxTokens(1024);
// @doctest id="5f14"
```

At this point, no HTTP call has been made. The facade holds an `InferenceRequestBuilder` that accumulates parameters. Every `with*()` call returns a new immutable copy, so the original instance is never modified.

### 2. Creating a Pending Handle

Calling `create()` (or a shortcut like `get()` or `response()`) builds the `InferenceRequest` and passes it to the runtime:

```php theme={null}
$pending = $inference->create();
// @doctest id="2d82"
```

The `InferenceRuntime` wraps the request in an `InferenceExecution` object and returns a `PendingInference` handle. Execution is still deferred -- no HTTP call has been sent yet.

The `InferenceExecution` tracks the full lifecycle state: the original request, retry attempts, usage accumulation, and the final response.

### 3. Triggering Execution

The HTTP call is triggered only when you read from the `PendingInference`:

```php theme={null}
$text = $pending->get();          // triggers execution, returns content string
$response = $pending->response(); // triggers execution, returns InferenceResponse
$stream = $pending->stream();     // triggers execution (streaming mode)
// @doctest id="11e8"
```

Internally, `PendingInference` delegates to `InferenceExecutionSession`, which orchestrates the full lifecycle.

### 4. The Execution Session

The `InferenceExecutionSession` is the heart of the lifecycle. It owns the `InferenceExecution` and reconciles the two ways a caller can ask for a result -- `response()` and `stream()` -- and delegates the rest to two collaborators, each built once per execution in its constructor:

| Collaborator                | Owns                                                                                                                  |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `InferenceLifecycleEmitter` | every lifecycle event, both stopwatches, the attempt counter, and the telemetry correlation stamped onto each attempt |
| `InferenceRetryLoop`        | the attempt budget, the length-recovery budget, the backoff delay, and the rewritten request a length recovery sends  |

They are per execution, never per attempt and never per delta -- the emitter in particular resolves its six listener gates once, in its own constructor, and constructing it on the delta path would put six `hasListenersFor()` calls behind every chunk.

There is no third collaborator holding a response cache, and that is deliberate. Repeated `response()` calls return the identical instance already, because the session reads it back off the `InferenceExecution` -- which the retry loop has updated with `withSuccessfulAttempt()` before the call can return. A separate cache keyed on `ResponseCachePolicy` existed and was unreachable: the execution check ran first and always won. `ResponseCachePolicy` keeps its meaning one layer down, where `BaseInferenceRequestDriver` maps it onto `StreamCachePolicy` to decide whether the HTTP stream is replayable.

Two methods on the session are worth knowing by name, because between them they are the only places an execution ends: `succeed()` and `terminate()`. `terminate()` takes an explicit `$throw` flag, which is the difference between a caller driving the execution (throws) and a stream callback invoked from inside the stream's own iteration (stores the error for the next `response()` call to rethrow).

It performs these steps for a non-streaming request:

1. **Dispatches `InferenceStarted`** -- signals the beginning of the operation, including the execution ID, request details, and whether streaming is enabled
2. **Dispatches `InferenceAttemptStarted`** -- signals the beginning of an attempt with the attempt number and model
3. **Calls the driver** -- `driver->makeResponseFor($request)` triggers the full request-response cycle:
   * The driver's request adapter converts `InferenceRequest` into an `HttpRequest`
   * The HTTP client sends the request to the provider
   * The driver's response adapter normalizes the raw `HttpResponse` into an `InferenceResponse`
4. **Checks the response** -- if the finish reason indicates a failure (error, content filter, or length limit), the session handles it according to the retry policy
5. **Dispatches success events**:
   * `InferenceResponseCreated` -- the response is ready
   * `InferenceAttemptSucceeded` -- the attempt completed, including finish reason and usage
   * `InferenceUsageReported` -- token usage (`InferenceUsage`) is reported with the model name
   * `InferenceCompleted` -- the entire operation is done, including total attempt count and timing
6. **Returns `InferenceResponse`** to the caller

Cost calculation is performed externally using a `FlatRateCostCalculator` with `InferencePricing` data from the `LLMConfig`, rather than being attached to the usage object in the pipeline.

### 5. Retry Handling

If the request fails with a retryable error (transient HTTP status, timeout, network error, or provider-classified retriable exception), the session:

1. Records the failure on the execution object
2. **Dispatches `InferenceAttemptFailed`** -- with the error details, HTTP status code, partial usage, and `willRetry: true`
3. Waits for the configured delay (exponential backoff with optional jitter)
4. **Dispatches a new `InferenceAttemptStarted`** and retries

If all attempts are exhausted, the session dispatches `InferenceCompleted` with `isSuccess: false` and throws the terminal error.

**Length-limit recovery** has special handling. When a response finishes with `Length` as the finish reason and the retry policy allows length recovery, the session can:

* **`'continue'`** -- append the partial response as an assistant message, add a continuation prompt, and retry
* **`'increase_max_tokens'`** -- increase the `max_tokens` option by the configured increment and retry

This is independent of the regular retry count and controlled by `lengthMaxAttempts`.

### 6. Cached Context

If the request includes a `CachedInferenceContext`, the driver applies it before sending. Cached context allows you to pre-configure messages, tools, tool choice, and response format that are prepended to or merged with the request's own values. This is particularly useful for system prompts or shared tool definitions that remain constant across calls.

## Streaming Lifecycle

When streaming is enabled, the flow diverges after the HTTP request is sent:

1. `PendingInference::stream()` validates that streaming was requested, then creates an `InferenceStream`
2. The driver produces an iterable of `PartialInferenceDelta` objects from the SSE event stream via `driver->makeStreamDeltasFor($request)`
3. The `InferenceStream` tracks visibility state through a `VisibilityTracker` and yields only deltas with meaningful changes (filtering out empty or duplicate deltas)

```php theme={null}
$stream = $inference->withMessages(Messages::fromString('Hello'))->stream();

foreach ($stream->deltas() as $delta) {
    echo $delta->contentDelta;  // incremental text
}

$finalResponse = $stream->final();  // assembled InferenceResponse
// @doctest id="7901"
```

### Stream Events

The stream dispatches events as deltas arrive:

* **`StreamFirstChunkReceived`** -- when the first visible delta arrives, including the request start time for TTFC measurement
* **`PartialInferenceDeltaCreated`** -- for each visible delta
* **`InferenceResponseCreated`** -- when the stream finishes and the final response is assembled from accumulated state

When a stream fails, the session records the failure the same way a non-streamed attempt would -- `InferenceAttemptFailed` with the partial usage accumulated so far, then `InferenceCompleted` with `isSuccess: false` -- and the error is rethrown to the caller.

This holds regardless of *when* the failure lands. A connection dropped at handshake, before a single delta arrives, reports exactly what a mid-stream failure reports; the partial usage is simply zero. Anything counting failures from lifecycle events can rely on that.

### Stream Processing

The stream supports functional-style processing through `map()`, `reduce()`, and `filter()`:

```php theme={null}
// Map deltas to extracted values
$contents = $stream->map(fn($delta) => $delta->contentDelta);

// Reduce deltas into a single value
$fullText = $stream->reduce(fn($carry, $delta) => $carry . $delta->contentDelta, '');

// Filter deltas
$toolDeltas = $stream->filter(fn($delta) => $delta->toolName !== '');

// Collect all visible deltas
$allDeltas = $stream->all();
// @doctest id="27b4"
```

### Delta Callback

You can register a callback that fires for every visible delta:

```php theme={null}
$stream->onDelta(function (PartialInferenceDelta $delta): void {
    echo $delta->contentDelta;
});
// @doctest id="b2d6"
```

### Stream Finalization

Calling `final()` on a stream that has not been fully consumed will drain the remaining deltas first, ensuring the final response is complete. A stream can only be consumed once -- calling `deltas()` a second time throws a `LogicException`.

The final response assembled from the stream goes through the same event dispatch as a synchronous response.

## Embeddings Lifecycle

The embeddings lifecycle is simpler since streaming is not involved:

1. **`Embeddings` builds an `EmbeddingsRequest`** from the configured inputs, model, and options
2. **`create()` returns `PendingEmbeddings`** -- a lazy handle that holds the request, driver, and event dispatcher
3. **`get()` triggers execution**:
   * The driver's `handle()` method translates and sends the HTTP request
   * The driver decodes the provider payload and returns an `EmbeddingsResponse`
   * `EmbeddingsResponseReceived` is dispatched
4. **`EmbeddingsResponse` is returned** -- containing vectors and usage

```php theme={null}
$response = Embeddings::using('openai')
    ->withInputs(['Hello', 'World'])
    ->get();

$vectors = $response->vectors();   // Vector[]
$first = $response->first();       // first Vector
$usage = $response->usage();       // InferenceUsage
// @doctest id="79b9"
```

Retry logic is handled internally by `PendingEmbeddings` based on the `EmbeddingsRetryPolicy` attached to the request. The retry loop follows the same exponential backoff pattern as inference retries.

## Response Caching

A `PendingInference` executes at most once. Repeat calls to `response()` or `get()` return the result of that execution without another HTTP call, and a repeat call after a failure rethrows the original error rather than retrying.

That memoization is unconditional -- it comes from the finalized attempt held on the `InferenceExecution`, not from `ResponseCachePolicy`. `withResponseCachePolicy()` is accepted and carried on the request, but the session's own cache is currently never read, because the finalized-attempt check always answers first. Do not rely on the policy to change single-session behaviour; it does not. Tracked as `instructor-eexl.23`.

```php theme={null}
use Cognesy\Polyglot\Inference\Enums\ResponseCachePolicy;

$pending = $inference
    ->withMessages(Messages::fromString('Hello'))
    ->withResponseCachePolicy(ResponseCachePolicy::Memory)
    ->create();

$first = $pending->response();  // makes HTTP call
$second = $pending->response(); // returns cached response
// @doctest id="d225"
```

For streaming, the stream itself cannot be replayed -- calling `deltas()` a second time will throw a `LogicException`. However, `final()` always returns the assembled response, which is stored in the execution object.