Skip to content

feat(ci_visibility): add test_session telemetry metric with provider and auto_injected tags#17696

Draft
gnufede wants to merge 1 commit intomainfrom
telemetry/test-session-metric
Draft

feat(ci_visibility): add test_session telemetry metric with provider and auto_injected tags#17696
gnufede wants to merge 1 commit intomainfrom
telemetry/test-session-metric

Conversation

@gnufede
Copy link
Copy Markdown
Member

@gnufede gnufede commented Apr 23, 2026

Summary

  • Adds a new test_session count metric emitted once per test session with provider and auto_injected tags
  • Reduces cardinality versus including these tags on the per-test event_finished metric
  • Called from the pytest plugin's session finish path alongside record_session_finished

Test plan

  • Unit tests for record_test_session with provider and auto_injected tags
  • Unit test for unsupported CI (None provider, auto_injected=False)
  • Verify metric appears in telemetry intake

Motivation

Per CI Visibility telemetry RFC: a dedicated low-cardinality test_session metric allows tracking CI provider distribution and auto-injection without inflating event_created/event_finished tag cardinality.

🤖 Generated with Claude Code

…and auto_injected tags

Adds a new `test_session` count metric emitted once per session with
`provider` and `auto_injected` tags, reducing cardinality versus
including these tags on the per-test `event_finished` metric.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gnufede gnufede requested a review from a team as a code owner April 23, 2026 12:00
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Codeowners resolved as

ddtrace/testing/internal/pytest/plugin.py                               @DataDog/ci-app-libraries
ddtrace/testing/internal/telemetry.py                                   @DataDog/ci-app-libraries
tests/testing/internal/test_telemetry.py                                @DataDog/ci-app-libraries

@gnufede gnufede marked this pull request as draft April 23, 2026 12:02
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4c3d81d767

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +379 to +382
TelemetryAPI.get().record_test_session(
ci_provider_name=self.manager.env_tags.get(CITag.PROVIDER_NAME),
is_auto_injected=self.manager.is_auto_injected,
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Emit test_session metric only from xdist main process

In pytest-xdist runs, pytest_sessionfinish executes in every worker and in the controller process, but record_test_session is called before the if not self.is_xdist_worker guard. This makes the new test_session counter scale with worker count (e.g., 9 emissions for 8 workers) instead of once per logical test session, which skews the provider/auto-injection telemetry this metric is meant to represent.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant