Observability, Logging, and Telemetry

LLM calls fail differently from ordinary API calls. A request can return HTTP 200 and still produce data the application cannot use. A response can look reasonable to a person and still violate a PHP type, enum, date format, or business rule.

That means normal "request succeeded" logging is not enough. In a production InstructorPHP workflow, the useful question is not only whether the provider responded. It is whether the model response crossed the application boundary as valid data.

Working rule: traces tell you where time went. Events and logs tell you why a structured-output attempt was accepted, repaired, or rejected.

The production problem

Consider a support ticket extraction flow. The model receives messy human text and the application expects a strict incident object:

use DateTimeInterface;
use Symfony\Component\Validator\Constraints as Assert;

final readonly class Incident
{
    public function __construct(
        #[Assert\NotBlank]
        public string $service,

        #[Assert\Choice(choices: ['low', 'medium', 'high', 'critical'])]
        public string $severity,

        #[Assert\Choice(choices: ['open', 'triaged', 'closed'])]
        public string $status,

        #[Assert\DateTime(format: DateTimeInterface::ATOM)]
        public string $reportedAt,
    ) {}
}

A model might answer:

{
  "service": "Checkout",
  "severity": "urgent",
  "status": "needs attention",
  "reportedAt": "today at 09:10 UTC"
}

The provider call succeeded. The application contract did not.

If this is only visible as "LLM request completed", operations will not know whether routing failed because of the prompt, the model, the validator, retry exhaustion, a provider issue, or downstream business code.

What the loop should record

For structured-output workflows, record the parts that explain the decision:

Do not store sensitive prompts and customer data by default. Use redaction, summaries, correlation IDs, and retention rules. The goal is enough evidence to debug a bad workflow without turning logs into a second database of private input.

The four InstructorPHP surfaces

InstructorPHP exposes observability at different levels. They are complementary, not interchangeable.

The runnable verification example is here: Observe StructuredOutput events, logs, and telemetry.

That example uses a local fake inference driver. It intentionally returns an invalid response first and a corrected response second. This makes the validation failure and retry deterministic, so the example proves the mechanisms without depending on provider behavior.

Events: local truth

Use events when application code needs to react to what happened during a structured-output execution.

use Cognesy\Instructor\Events\Response\ResponseValidationFailed;
use Cognesy\Instructor\StructuredOutputRuntime;
use Cognesy\Polyglot\Inference\LLMProvider;

$runtime = StructuredOutputRuntime::fromProvider(
    LLMProvider::using('openai')->withModel('gpt-4o-mini'),
)
    ->onEvent(ResponseValidationFailed::class, function (ResponseValidationFailed $event): void {
        logger()->warning('incident_extraction.validation_failed', [
            'event' => $event->toArray(),
        ]);
    });

Use onEvent() when you know which event matters. Use wiretap() when you are instrumenting a workflow and want to see the full stream:

$runtime = $runtime->wiretap(function (object $event): void {
    logger()->debug('instructor.event', [
        'class' => $event::class,
    ]);
});

The verification example observed:

From an application perspective, this is the safest place to capture validation-specific evidence. If a business workflow needs to know that severity=urgent was rejected and repaired to critical, capture that from events or from a logging pipeline connected to events.

EventLog: durable JSONL

EventLog is useful when you want a process-local structured log without building a full logging stack first.

use Cognesy\Instructor\StructuredOutputRuntime;
use Cognesy\Logging\EventLog;
use Cognesy\Polyglot\Inference\LLMProvider;

EventLog::enable('/var/log/instructor/structured-output.jsonl');

$events = EventLog::root('incident-extraction');

$runtime = StructuredOutputRuntime::fromProvider(
    provider: LLMProvider::using('openai')->withModel('gpt-4o-mini'),
    events: $events,
);

The important detail is that EventLog::root() creates the dispatcher with the JSONL sink attached. If you pass a completely separate custom event dispatcher, the file sink will not see those events unless you attach logging to that dispatcher yourself.

In the verification example, JSONL logging captured nine structured entries for the invalid-then-corrected run. That is enough to reconstruct the lifecycle without adding feature-specific print statements.

Logging pipeline: application logs

Most production systems already have a log destination. InstructorPHP's logging pipeline lets you filter, format, and write event-derived log entries to a destination such as PSR-3, Monolog, Symfony, or Laravel.

use Cognesy\Instructor\Events\Attempt\NewValidationRecoveryAttempt;
use Cognesy\Instructor\Events\Response\ResponseValidationFailed;
use Cognesy\Instructor\Events\StructuredOutput\StructuredOutputResponseGenerated;
use Cognesy\Logging\Filters\EventClassFilter;
use Cognesy\Logging\Formatters\MessageTemplateFormatter;
use Cognesy\Logging\Pipeline\LoggingPipeline;
use Cognesy\Logging\Writers\PsrLoggerWriter;

$pipeline = LoggingPipeline::create()
    ->filter(new EventClassFilter(includedClasses: [
        ResponseValidationFailed::class,
        NewValidationRecoveryAttempt::class,
        StructuredOutputResponseGenerated::class,
    ]))
    ->format(new MessageTemplateFormatter([
        ResponseValidationFailed::class => 'structured output validation failed',
        NewValidationRecoveryAttempt::class => 'structured output retry requested',
        StructuredOutputResponseGenerated::class => 'structured output accepted',
    ]))
    ->write(new PsrLoggerWriter($logger))
    ->build();

$runtime = $runtime->wiretap($pipeline);

This keeps app code cleaner. The controller or queue job does not need to parse raw events. It receives a typed Incident or handles a deliberate failure, while the observability policy lives in one logging pipeline.

For business systems, this matters because the operational records match the workflow. A support leader can ask, "How many critical tickets needed repair before routing?" Engineering can answer from retry and validation logs instead of manually sampling prompts.

Telemetry: traces and spans

Telemetry is the right place to see execution shape: structured-output span, inference spans, attempts, HTTP spans for real provider traffic, duration, token usage where available, and exporter-specific traces.

InstructorPHP comes with ready telemetry integrations, including Logfire and Langfuse exporters. You can wire those into the same runtime event bus, or provide your own exporter when your organization already has a telemetry backend.

InstructorPHP wires telemetry by attaching a bridge to the same runtime event bus:

use Cognesy\Http\Telemetry\HttpClientTelemetryProjector;
use Cognesy\Instructor\Telemetry\InstructorTelemetryProjector;
use Cognesy\Polyglot\Telemetry\PolyglotTelemetryProjector;
use Cognesy\Telemetry\Application\Projector\CompositeTelemetryProjector;
use Cognesy\Telemetry\Application\Projector\RuntimeEventBridge;
use Cognesy\Telemetry\Application\Registry\TraceRegistry;
use Cognesy\Telemetry\Application\Telemetry;

$telemetry = new Telemetry(
    registry: new TraceRegistry(),
    exporter: $exporter,
);

(new RuntimeEventBridge(new CompositeTelemetryProjector([
    new InstructorTelemetryProjector($telemetry),
    new PolyglotTelemetryProjector($telemetry),
    new HttpClientTelemetryProjector($telemetry),
])))->attachTo($events);

The repository includes runnable examples for Logfire and Langfuse exporters, and the local verification example uses a small recording exporter to prove projection without external credentials.

The evidence matters here. In the deterministic validation run, telemetry exported:

llm.inference.attempt
llm.inference
llm.inference.attempt
llm.inference
structured_output.execute

It did not export a separate structured_output.response_validation_failed telemetry observation in that local run, even though the validation failure was visible through events and logs.

That is a useful operating constraint: use telemetry for execution and provider traces; use events/logging when you need detailed validation and repair evidence. Do not assume every domain-level failure detail will appear as a standalone trace observation unless you have verified that path in your runtime configuration.

What this means operationally

A practical production setup usually looks like this:

1. Use telemetry for trace shape, latency, provider calls, attempts, and cross-service correlation.
2. Use event callbacks or a logging pipeline for validation failures, retry reasons, and accepted object summaries.
3. Use JSONL EventLog during development, incident analysis, or controlled process-level auditing.
4. Redact raw input and output before they leave the application boundary.
5. Count repair attempts separately from first-pass success.

The business value is not "more logs." The value is being able to explain why an AI-assisted workflow made a decision, how often it needed repair, and where failures are coming from.

If support routing, claim intake, CRM enrichment, billing review, or compliance screening relies on InstructorPHP output, the observability layer should answer three questions quickly:

That split keeps the system understandable. Traces show the execution path. Logs explain the boundary decision. Application code stays focused on business behavior instead of carrying one-off debugging logic through every feature.