feat(ai): add embeddings instrumentation and CI compatibility by constantinius · Pull Request #1111 · getsentry/sentry-laravel

constantinius · 2026-03-16T14:31:51Z

Introduce embeddings span tracing and complete the remaining AI integration plumbing, including laravel/ai dependency, CI matrix handling, and phpunit configuration so AI tests run only in compatible environments. This final stacked branch brings the feature set to parity with the original large PR.

Summary

Finalizes the AI integration stack by adding embeddings tracing and the remaining compatibility/CI plumbing needed for matrix stability.

Included

Adds embeddings instrumentation:
- GeneratingEmbeddings -> start gen_ai.embeddings span
- EmbeddingsGenerated -> finish/enrich embeddings span
- embeddings input truncation helper
Adds tracing.gen_ai_embeddings toggle.
Adds/updates compatibility and CI wiring:
- composer.json (laravel/ai dev dependency)
- .github/workflows/ci.yaml
- .github/workflows/cs.yaml
- phpunit.xml
Includes final AI test adjustments to align with version/runtime constraints.

Made-with: Cursor

Issues

Contributes to https://linear.app/getsentry/issue/TET-2064/split-current-pr
Contributes to https://linear.app/getsentry/issue/TET-1971/laravel-ai-integration

Add the initial AI integration wiring that records invoke_agent spans for prompting and prompted lifecycle events, including model/provider and token usage metadata. This establishes the stack base without chat, tool, message, or embeddings instrumentation. Made-with: Cursor

linear-code · 2026-03-16T14:34:15Z

TET-2064 Split current PR

TET-1971 Laravel AI integration

sentry · 2026-03-16T14:35:06Z

src/Sentry/Laravel/Features/AiIntegration.php

+        }
+
+        foreach ($source->toolCalls as $toolCall) {
+            $parts[] = $this->buildToolCallPart($toolCall);


Bug: The buildOutputMessages method iterates over $source->toolCalls and $source->toolResults without checking if they are null, which can cause a TypeError.
_{Severity: CRITICAL}

Suggested Fix

Add null-coalescing operators to the foreach loops to ensure an empty array is used as a default value if the properties are null. For example: foreach (($source->toolCalls ?? []) as $toolCall) and foreach (($source->toolResults ?? []) as $toolResult).

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: src/Sentry/Laravel/Features/AiIntegration.php#L738 Potential issue: In the `buildOutputMessages` method, the properties `$source->toolCalls` and `$source->toolResults` are iterated over using `foreach` without a prior null check. These properties are declared on their respective objects but are not initialized in the constructor, meaning they can be `null`. If the Laravel AI library returns a response or step object where these properties are not explicitly set, attempting to iterate over `null` will cause a `TypeError` in PHP 8, leading to a runtime crash.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

✅ Fixed: CI runs AI tests on PHP versions lacking required dependencies
- The CI AI test step now only runs for Laravel 12 on PHP 8.4/8.5, matching the conditional laravel/ai installation guard.
✅ Fixed: Nullable $invocationId risks undefined array key access
- The method contract was tightened by changing setConversationIdOnSpans to require a non-null string invocation ID before indexing the invocation map.

Or push these changes by commenting:

@cursor push 026cff2827

Preview (026cff2827)

diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
--- a/.github/workflows/ci.yaml
+++ b/.github/workflows/ci.yaml
@@ -79,7 +79,7 @@
         run: vendor/bin/phpunit --coverage-clover=coverage.xml --coverage-filter=src/Sentry
 
       - name: Run AI integration tests
-        if: matrix.packages.laravel == '^12.0'
+        if: matrix.packages.laravel == '^12.0' && (matrix.php == '8.4' || matrix.php == '8.5')
         run: vendor/bin/phpunit test/Sentry/Features/AiIntegrationTest.php
 
       - name: Upload coverage to Codecov

diff --git a/src/Sentry/Laravel/Features/AiIntegration.php b/src/Sentry/Laravel/Features/AiIntegration.php
--- a/src/Sentry/Laravel/Features/AiIntegration.php
+++ b/src/Sentry/Laravel/Features/AiIntegration.php
@@ -573,7 +573,7 @@
         }
     }
 
-    private function setConversationIdOnSpans(?string $invocationId, ?string $conversationId): void
+    private function setConversationIdOnSpans(string $invocationId, ?string $conversationId): void
     {
         if ($conversationId !== null) {
             $invocation = $this->invocations[$invocationId];

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

.github/workflows/ci.yaml

cursor · 2026-03-16T14:37:46Z

src/Sentry/Laravel/Features/AiIntegration.php

+                $span->setData($data);
+            }
+        }
+    }


Nullable $invocationId risks undefined array key access

Low Severity

setConversationIdOnSpans declares $invocationId as ?string but accesses $this->invocations[$invocationId] without a null guard when $conversationId is non-null. If $invocationId is ever null, this produces an undefined array key warning. The parameter type hints nullable but the method body assumes non-null — it likely just needs to be typed string.

composer.json

Replace inline stub definitions for Laravel AI interfaces, classes, events, attributes, and file types with the real implementations from the laravel/ai package added as a dev dependency. Drop PHP version and Laravel version skip guards from setUp() since these tests only run on PHP 8.4+. Simplify TestAgentWithConfig to use native PHP 8 attributes directly instead of the eval() workaround for older PHP versions.

sentry · 2026-04-02T13:15:18Z

src/Sentry/Laravel/Features/AiIntegration.php

+            }
+            $entryBytes = \strlen($inputJson) + (empty($kept) ? 0 : 1); // +1 for comma separator
+


Bug: The truncateEmbeddingInputs function iterates backward and breaks early, potentially discarding smaller inputs that would fit within the budget, leading to unnecessary data loss.
_{Severity: HIGH}

Suggested Fix

Modify the truncateEmbeddingInputs function to not break when an item is too large. Instead, it should continue to the next item in the backward iteration. This will allow it to check earlier, potentially smaller items in the array and include them if they fit within the budget, maximizing data retention.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: src/Sentry/Laravel/Features/AiIntegration.php#L1063-L1065 Potential issue: The `truncateEmbeddingInputs` function has incorrect logic for truncating inputs based on a byte budget. It iterates backward from the end of the input array and stops processing entirely (`break`) as soon as it encounters an item that is too large to fit in the remaining budget. This prevents the function from considering smaller items that may appear earlier in the array and would fit. This can lead to significant and silent data loss by underutilizing the available budget, as inputs that should have been included are unnecessarily discarded.

Remove laravel/ai globally in both phpunit and phpunit-legacy jobs since it requires PHP 8.4+ and only supports Laravel 12. Re-add it conditionally for the supported combination (PHP 8.4+, Laravel 12).

cursor · 2026-04-02T13:34:35Z

src/Sentry/Laravel/Features/AiIntegration.php

+
+            array_unshift($kept, $inputs[$i]);
+            $totalBytes += $entryBytes;
+        }


Embeddings truncation keeps last inputs instead of first

Medium Severity

truncateEmbeddingInputs iterates backwards from the end of the $inputs array, prioritizing keeping the last inputs and dropping the first when the byte budget is exceeded. The Python SDK equivalent keeps the first inputs (earliest in the list). Additionally, the fallback on line 1075 uses end($inputs) to grab the last input, which is consistent with the backwards logic but inconsistent with the expected behavior of preserving the beginning of the input list. This means when truncation kicks in, the most important (first-provided) embedding inputs are silently dropped.

Additional Locations (1)

src/Sentry/Laravel/Features/AiIntegration.php#L1073-L1080

The Laravel AI classes are not available when PHPStan runs on PHP 8.3 without laravel/ai installed. Add baseline ignores matching the existing pattern used for other optional packages (Sanctum, Livewire, Lumen).

Capture provider HTTP request/response events as gen_ai.chat spans under invoke_agent and enrich each chat span with response model, usage, finish reason, and conversation id mapping. This keeps tool/message/embeddings behavior out of scope for follow-up stacked branches. Made-with: Cursor

Add execute_tool span lifecycle around InvokingTool and ToolInvoked events, and propagate serialized tool definitions onto invoke_agent and chat spans. This isolates tool-level monitoring while leaving message payload capture and embeddings for later stacked branches. Made-with: Cursor

Add a HasTools stub and implement it on TestAgent so tool definition metadata is populated under the same contract checks used by AiIntegration. Made-with: Cursor

Add structured input/output message capture for invoke_agent and chat spans, including attachment transformation, payload truncation, and binary redaction safeguards. This layers privacy-aware message instrumentation on top of existing invoke/chat/tool tracing without introducing embeddings yet. Made-with: Cursor

Pass attachments as an array to match Laravel AI AgentPrompt constructor typing on newer framework versions. Made-with: Cursor

Introduce embeddings span tracing and complete the remaining AI integration plumbing, including laravel/ai dependency, CI matrix handling, and phpunit configuration so AI tests run only in compatible environments. This final stacked branch brings the feature set to parity with the original large PR. Made-with: Cursor

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

cursor · 2026-04-02T13:53:58Z

src/Sentry/Laravel/Features/AiIntegration.php

+                    $inputMessages = $this->buildChatInputMessages($steps, $index);
+                } else {
+                    // Streaming without steps: use response output as input for subsequent chat spans
+                    $inputMessages = $this->buildOutputMessages($response);


Empty Collection check always evaluates true, dead streaming fallback

Medium Severity

$steps obtained from $response->steps ?? [] is typically a Laravel Collection. PHP's empty() on any object — including an empty Collection — always returns false. So !empty($steps) at line 553 is always true when $steps is a Collection, making the streaming fallback branch (lines 555–557) unreachable dead code. For streaming responses with multiple chat spans and PII enabled, subsequent spans get empty input messages instead of the intended response output. Using count($steps) > 0 or $steps->isNotEmpty() would fix this.

Additional Locations (1)

src/Sentry/Laravel/Features/AiIntegration.php#L517-L518

sentry bot reviewed Mar 16, 2026

View reviewed changes

constantinius requested a review from Litarnus March 16, 2026 14:35

cursor bot reviewed Mar 16, 2026

View reviewed changes

constantinius changed the base branch from constantinius/feat/ai/integration to constantinius/feat/ai/integration-base March 16, 2026 14:53

constantinius added 4 commits March 17, 2026 08:54

fix(ai): compatibility with PHP < 8 and test class fix

e5ea71e

fix(ai): test fix

1c95e55

fix(ai): test fix

93f3f58

fix(ai): really fixing test issue

cd9b98b

cursor bot reviewed Mar 17, 2026

View reviewed changes

composer.json Outdated Show resolved Hide resolved

constantinius force-pushed the constantinius/feat/ai/embeddings-and-polish branch 2 times, most recently from 6bfe4a7 to 7cc0d90 Compare April 2, 2026 13:11

sentry bot reviewed Apr 2, 2026

View reviewed changes

ci: handle laravel/ai dev dependency in CI matrix

6f06a2c

Remove laravel/ai globally in both phpunit and phpunit-legacy jobs since it requires PHP 8.4+ and only supports Laravel 12. Re-add it conditionally for the supported combination (PHP 8.4+, Laravel 12).

constantinius force-pushed the constantinius/feat/ai/embeddings-and-polish branch from 7cc0d90 to f432815 Compare April 2, 2026 13:31

cursor bot reviewed Apr 2, 2026

View reviewed changes

constantinius added 11 commits April 2, 2026 15:45

ci: add PHPStan baseline entries for optional laravel/ai types

94cf6f2

The Laravel AI classes are not available when PHPStan runs on PHP 8.3 without laravel/ai installed. Add baseline ignores matching the existing pattern used for other optional packages (Sanctum, Livewire, Lumen).

ci: add PHPStan baseline entries for chat span types

4b1fa6a

fix(ai): align tool definition test with HasTools gating

4c89ea0

Add a HasTools stub and implement it on TestAgent so tool definition metadata is populated under the same contract checks used by AiIntegration. Made-with: Cursor

ci: add PHPStan baseline entries for tool span types

1e34147

fix(ai): use array attachments in attachment handling test

7d9054f

Pass attachments as an array to match Laravel AI AgentPrompt constructor typing on newer framework versions. Made-with: Cursor

ci: add PHPStan baseline entries for message redaction types

1c5d398

ci: add PHPStan baseline entries for embeddings types

1dd7a83

constantinius force-pushed the constantinius/feat/ai/embeddings-and-polish branch from f432815 to 1dd7a83 Compare April 2, 2026 13:49

cursor bot reviewed Apr 2, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(ai): add embeddings instrumentation and CI compatibility#1111

feat(ai): add embeddings instrumentation and CI compatibility#1111
constantinius wants to merge 18 commits intoconstantinius/feat/ai/integration-basefrom
constantinius/feat/ai/embeddings-and-polish

constantinius commented Mar 16, 2026 •

edited

Loading

Uh oh!

linear-code bot commented Mar 16, 2026

Uh oh!

sentry bot Mar 16, 2026

Uh oh!

cursor bot left a comment •

edited

Loading

Uh oh!

Uh oh!

cursor bot Mar 16, 2026

Uh oh!

Uh oh!

sentry bot Apr 2, 2026

Uh oh!

cursor bot Apr 2, 2026

Uh oh!

cursor bot left a comment

Uh oh!

cursor bot Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		}
		$entryBytes = \strlen($inputJson) + (empty($kept) ? 0 : 1); // +1 for comma separator

Uh oh!

Conversation

constantinius commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Included

Issues

Uh oh!

linear-code bot commented Mar 16, 2026

Uh oh!

sentry bot Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor bot Mar 16, 2026

Choose a reason for hiding this comment

Nullable $invocationId risks undefined array key access

Uh oh!

Uh oh!

sentry bot Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

cursor bot Apr 2, 2026

Choose a reason for hiding this comment

Embeddings truncation keeps last inputs instead of first

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Apr 2, 2026

Choose a reason for hiding this comment

Empty Collection check always evaluates true, dead streaming fallback

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

constantinius commented Mar 16, 2026 •

edited

Loading

cursor bot left a comment •

edited

Loading

Nullable `$invocationId` risks undefined array key access