Skip to content

DI: add re-entrancy guard for method probes on stdlib#5560

Draft
p-datadog wants to merge 32 commits intomasterfrom
di-integration-test-stdlib-probes
Draft

DI: add re-entrancy guard for method probes on stdlib#5560
p-datadog wants to merge 32 commits intomasterfrom
di-integration-test-stdlib-probes

Conversation

@p-datadog
Copy link
Copy Markdown
Member

@p-datadog p-datadog commented Apr 3, 2026

What does this PR do?

Adds a fiber-local re-entrancy guard to method probes and integration tests exercising it.

When a probe targets a stdlib method that DI calls internally during snapshot building (e.g., String#length via SecureRandom.uuid), the probe would re-enter DI processing recursively. The guard prevents this by skipping DI processing for calls that occur while DI is already processing a probe.

The guard is a split guard: held during DI pre-processing (serialization) and post-processing (notification), but released during super() so that recursive calls and nested probes fire normally during user code execution.

Motivation:

Without this, a non-capture probe on String#length causes SystemStackError via: probe fires → build_snapshot → SecureRandom.uuid → gen_random_urandom → String#length → probe fires → ...

Change log entry

Yes. Dynamic Instrumentation: fix SystemStackError when probes target standard library methods used internally by probe processing.

Additional Notes:

The guard uses Thread.current[:datadog_di_in_probe] (fiber-local, not thread-local) so each fiber tracks independently — correct for fiber-based web servers (Falcon, async).

Line probes are not affected because Ruby's TracePoint is self-disabling during its callback.

How to test the change?

Automated:

bundle exec rspec spec/datadog/di/instrumenter_spec.rb spec/datadog/di/integration/instrumentation_spec.rb spec/datadog/di/integration/stdlib_probe_spec.rb

135 examples, 0 failures.

Gobo (manual verification via Live Debugger UI):

  1. Deploy gobo with a dd-trace-rb that includes this PR's branch
  2. In the Datadog Live Debugger UI, set a method probe on String class, length method — this is the C-implemented stdlib method that triggered the SystemStackError
  3. Set the probe to non-capture (log probe without snapshot) — this is the configuration that caused the crash due to the 5000/sec rate limit allowing recursive invocations
  4. Hit any gobo endpoint (e.g., the home page) — String#length is called pervasively by Ruby code
  5. Expected with this PR: a snapshot appears in the Live Debugger UI; gobo responds normally
  6. Expected without this PR: gobo crashes with SystemStackError: stack level too deep

p-ddsign and others added 28 commits March 27, 2026 14:13
When CodeTracker starts, use the all_iseqs C extension to populate the
registry with instruction sequences for files that were loaded before
tracking began. This enables line probes on third-party code and
application code loaded at boot time.

Only whole-file iseqs (first_lineno == 0) are backfilled — per-method
iseqs require instrumenter changes to select the correct iseq for a
target line and will be supported in a follow-up.

Backfill does not overwrite entries from :script_compiled, which are
authoritative. The C extension availability is checked via
DI.respond_to?(:all_iseqs) so the code gracefully degrades when
the extension is not compiled.

- Added CodeTracker#backfill_registry
- Called from CodeTracker#start after trace point is enabled
- Added RBS signature
- Added tests for backfill behavior and C extension fallback

Co-Authored-By: Claude <noreply@anthropic.com>
- Added rescue block around backfill_registry so failures are
  best-effort (logged + telemetry) rather than propagating
- Replaced all skip-based tests with mock-based tests that exercise
  backfill logic without requiring the compiled C extension
- Added tests for: mixed iseq types, multiple files, error handling,
  suffix/exact lookup on backfilled entries, start ordering
- 27 examples, 0 failures, 0 pending, 0 skipped

Co-Authored-By: Claude <noreply@anthropic.com>
Tests the end-to-end flow: test class loaded before code tracking
starts → CodeTracker#start triggers backfill via all_iseqs C extension
→ iseq recovered from object space → line probe installed on
backfilled iseq → probe fires and captures local variables.

Runs under rake spec:di_with_ext (requires compiled C extension).

Three test cases:
- Probe installs successfully on backfilled iseq
- Probe fires when target line executes
- Snapshot captures local variables from backfilled iseq

Co-Authored-By: Claude <noreply@anthropic.com>
On macOS CI the C extension is compiled, so backfill_registry populates
the CodeTracker registry with pre-loaded files during start. This broke
existing tests that expect the registry to be empty after start or to
contain exactly N explicitly-loaded files.

Fix by stubbing backfill_registry in test contexts that exercise
:script_compiled behavior. Backfill is tested separately in its own
describe blocks.

Affected contexts:
- CodeTracker #start (before block)
- CodeTracker shared context 'when code tracker is running'
- CodeTracker #iseqs_for_path_suffix (around block)
- Instrumenter shared context 'with code tracking'

Co-Authored-By: Claude <noreply@anthropic.com>
…kfill

The backfill filter used first_lineno == 0 to identify whole-file iseqs,
but most whole-file iseqs from all_iseqs have first_lineno == 1. The new
DI.iseq_type method reads the iseq type directly from the Ruby VM struct
and returns a symbol (:top, :method, :block, :class, etc.).

The backfill now filters by type == :top || type == :main, which
correctly identifies whole-file iseqs regardless of first_lineno.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rb_iseq_type is an internal Ruby function that only exists in Ruby 3.1+.
On Ruby 2.7 and 3.0, referencing it causes an undefined symbol error at
load time, crashing the entire C extension (including all_iseqs and
exception_message which work fine on those versions).

Use have_func in extconf.rb to detect rb_iseq_type at compile time, and
wrap the iseq_type function + registration in #ifdef HAVE_RB_ISEQ_TYPE.
The Ruby code in code_tracker.rb already handles the missing method via
DI.respond_to?(:iseq_type) with a first_lineno fallback.

Co-Authored-By: Claude <noreply@anthropic.com>
… require

- Add doc comments for rb_iseqw_new and rb_iseqw_to_iseq prototypes in di.c
  (internal Ruby functions used without documentation)
- Add error handling test coverage for backfill_registry: verify logger.debug
  is called with the error message and telemetry.report is called when
  DI.current_component is available
- Add test coverage for the first_lineno == 0 fallback path when iseq_type
  is unavailable (Ruby versions without rb_iseq_type)
- Add missing require "datadog/di/spec_helper" to iseq_type_spec.rb for
  consistency with other ext specs
- Fix skip message: iseq_type availability depends on rb_iseq_type in the
  Ruby runtime, not on the DI C extension

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- di.c: Document that rb_iseq_type was added in Ruby 3.1, explain the
  HAVE_RB_ISEQ_TYPE compile-time guard, and note the fallback path
- code_tracker.rb: Replace "first_lineno == 0" YARD doc with full
  description of both strategies (iseq_type on 3.1+, first_lineno
  heuristic on older Rubies) and their tradeoffs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The YARD doc claimed the first_lineno == 0 fallback "can match top-level
eval iseqs" but this is wrong. InstructionSequence.compile passes
first_lineno = 1 (not 0), and require/load passes INT2FIX(0) in Ruby's
rb_iseq_new_top/rb_iseq_new_main. Both strategies produce the same
result in practice.

Verified by reading Ruby 3.0 source (iseq.c lines 813-822):
  rb_iseq_new_with_opt(ast, name, path, realpath, INT2FIX(0), ...)
  → ISEQ_TYPE_TOP with first_lineno = 0

And compile path (iseq.c line 1064):
  rb_iseq_new_with_opt(&ast->body, label, file, realpath, line, ...)
  → line defaults to INT2FIX(1) for compile/eval

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verify idempotency: calling backfill_registry a second time with the
same iseqs doesn't duplicate entries (registry.key? guard).

Also verify that a second call with new iseqs adds them without
overwriting entries from the first call.

Co-Authored-By: Claude <noreply@anthropic.com>
The guard was purely defensive — the C extension is always compiled
when DI is active (enforced by environment_supported? in component.rb).
The rescue block at the bottom of backfill_registry already catches any
exception if file_iseqs fails, making the guard redundant.

Co-Authored-By: Claude <noreply@anthropic.com>
The method is called for side effects only. Without the explicit nil,
the happy path leaked the synchronize return value and the rescue path
leaked the telemetry report return value.

Co-Authored-By: Claude <noreply@anthropic.com>
On older Rubies, accessing an uninitialized instance variable via &.
produces a warning: "instance variable @current_components not
initialized". This triggers loading_spec failures because
datadog/di/preload produces unexpected output.

The variable is accessed by DI.current_component (called from
backfill_registry's error boundary) before any component is added.
Initializing to nil at module level suppresses the warning while
preserving the existing lazy-init behavior in add_current_component.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
RSpec's verify_partial_doubles rejects allow(DI).to receive(:iseq_type)
when the method doesn't exist on the module. On Ruby < 3.1,
rb_iseq_type is not available so DI.iseq_type is never defined.

Fix: conditionally stub iseq_type only when it exists. On older Rubies,
let respond_to?(:iseq_type) return false naturally and exercise the
first_lineno == 0 fallback path — which is what production does.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The pre-loaded test class's iseq can be garbage collected before
backfill walks the object space, causing DITargetNotInRegistry. In
production, application code is referenced by live constants/methods
and survives GC. In the test, the iseq is more ephemeral.

Disable GC around activate_tracking! (which calls backfill_registry)
to ensure the iseq is still in the object space when all_iseqs runs.
Re-enable immediately after.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two root causes:

1. code_tracker_spec.rb: iseq_type was stubbed with and_call_original,
   but the C function expects a real RubyVM::InstructionSequence, not a
   test double. Stub returns :top for first_lineno==0, :method otherwise.

2. backfill_integration_spec.rb: The top-level file iseq (first_lineno=0,
   type=:top) is not referenced by any constant or method after loading.
   GC could collect it between require_relative (file load time) and the
   before block's backfill_registry call. Move GC.disable to file level,
   immediately before require_relative, so the iseq survives until
   backfill walks the object space.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
The ivar is initialized to nil to avoid Ruby 2.6/2.7 warnings.
RBS type needs to reflect this. Silence false positive on <<
after ||= (Steep doesn't track that ||= guarantees non-nil).

Co-Authored-By: Claude <noreply@anthropic.com>
Root cause: the top-level (:top) iseq for the test class file has no
references after loading completes — only class/method child iseqs
survive via BackfillIntegrationTestClass. The previous approach disabled
GC at file load time and re-enabled it in the before block after
backfill. This protected the first test, but after deactivate_tracking!
cleared the registry (the only reference to the iseq), GC could collect
it before the next test's backfill_registry walked object space.

Fix: capture the top-level iseq in a constant (BACKFILL_TEST_TOP_ISEQ)
immediately after loading, before GC can collect it. The constant keeps
the iseq alive for the lifetime of the process, so backfill_registry
can find it in every test regardless of GC activity.

Verified: 0 failures across 8 consecutive full DI suite runs (714
examples each), vs ~20% failure rate before the fix.

Co-Authored-By: Claude <noreply@anthropic.com>
Add "Iseq Lifecycle and GC" section to DynamicInstrumentationDevelopment.md
covering: iseq types created on file load, which survive GC and why,
implications for backfill_registry, and the correct test pattern for
keeping top-level iseqs alive across tests.

Add cross-reference from code_tracker.rb backfill_registry docstring.

Co-Authored-By: Claude <noreply@anthropic.com>
When a file's whole-file (:top) iseq has been garbage collected,
per-method iseqs from all_iseqs can still be used to target line
probes. This covers 86% of files that were previously untargetable.

Changes:
- backfill_registry stores per-method iseqs in per_method_registry
  (grouped by path) instead of discarding them
- New iseq_for_line(suffix, line) method tries whole-file iseq first,
  then searches per-method iseqs for one whose trace_points include
  the target line
- Instrumenter uses iseq_for_line when available, falls back to
  iseqs_for_path_suffix for compatibility

Verified: 37 code_tracker tests pass, lint clean, types clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Loads a test class, GCs the top iseq, then verifies that the backfill
finds the surviving method iseq and a line probe can be installed,
fired, and captures local variables through it.

Precondition checks skip the test if GC didn't collect the top iseq
or if the C extension is unavailable.

Verified: 3 integration tests pass (install, fire, capture locals).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The throwable now includes a stacktrace array (from the C extension
commit). Also update error message assertion for the new
raise_if_probe_in_loaded_features format.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The raise_if_probe_in_loaded_features now reports whether per-method
iseqs exist or not, instead of the generic "not in code tracker
registry" message.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Distinguish between "has per-method iseqs but none cover this line"
and "has no surviving iseqs at all". Include the target line number
in the error. Helps users understand why a line probe failed and
whether the file is partially targetable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The method gained line_no and code_tracker parameters in the per-method
iseq support commit, but the RBS declaration was not updated.

Co-Authored-By: Claude <noreply@anthropic.com>
On Ruby < 3.1, DI.iseq_type is not defined (the C extension gates it
with have_func for rb_iseq_type). RSpec's partial double verification
rejects stubs for undefined methods, causing 10 failures in the
iseq_for_line and per-method iseq tests.

Fix: guard iseq_type stubs with `if Datadog::DI.respond_to?(:iseq_type)`,
matching the existing pattern in backfill_registry tests (lines 246-255).
On Ruby < 3.1, the code under test falls back to first_lineno == 0.

Co-Authored-By: Claude <noreply@anthropic.com>
…essing

Test DI's behavior when probes target standard library methods that DI
itself calls internally (re-entrancy scenarios). Key findings:

- Capture probes (rate limit 1/sec) on String#length, Hash#each,
  Array#map, Object#instance_variables: rate limiter prevents infinite
  recursion — nested invocations from DI processing are rate-limited
  and call the original method via super.

- Non-capture probes (rate limit 5000/sec) on String#length:
  SystemStackError — the high rate limit allows recursive invocations
  via SecureRandom.uuid -> gen_random_urandom -> String#length -> probe
  -> build_snapshot -> SecureRandom.uuid -> ... The stack overflow
  occurs inside rate_limiter.allow? before the rate limit can stop it.

- Line probe on Set#include? (set.rb): works with untargeted trace
  points since set.rb is loaded before code tracking starts.

- Method probe on Module#prepend: fires during its own installation
  but does not cause infinite recursion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@p-datadog p-datadog added the AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos label Apr 3, 2026
@github-actions github-actions bot added the dev/testing Involves testing processes (e.g. RSpec) label Apr 3, 2026
p-ddsign and others added 3 commits April 3, 2026 16:27
… vulnerability

Line probes (TracePoint) are NOT vulnerable to re-entrancy because Ruby
self-disables a TracePoint during its callback. Method probes (module
prepending) ARE vulnerable because prepend has no such protection.

New tests:
- Set#include? method probe with capture: safe (rate limiter)
- Set#include? method probe without capture: safe (not in snapshot path)
- Set#include? line probe without capture: safe (TracePoint self-disabling)

Updated file header with comprehensive findings summary.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents the desired behavior that probes on stdlib methods should
only fire for customer code, not DI-internal processing (add_probe,
serialization, snapshot building, flush, shutdown).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevents SystemStackError when probes target stdlib methods that DI
calls internally (e.g., String#length via SecureRandom.uuid during
snapshot building).

The guard uses fiber-local storage (Thread.current[:datadog_di_in_probe])
and is held only during DI processing phases (pre-processing and
post-processing), released during super() so recursive calls and
nested probes on other methods fire normally during user code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@p-datadog p-datadog changed the title DI: integration tests for probes on stdlib methods used by DI processing DI: add re-entrancy guard for method probes on stdlib Apr 3, 2026
@p-datadog p-datadog changed the base branch from di-per-method-iseq to master April 3, 2026 20:57
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 3, 2026

Typing analysis

Note: Ignored files are excluded from the next sections.

steep:ignore comments

This PR introduces 4 steep:ignore comments, and clears 2 steep:ignore comments.

steep:ignore comments (+4-2)Introduced:
lib/datadog/di/base.rb:109
lib/datadog/di/instrumenter.rb:265
lib/datadog/di/instrumenter.rb:381
lib/datadog/di/instrumenter.rb:383
Cleared:
lib/datadog/di/instrumenter.rb:233
lib/datadog/di/instrumenter.rb:341

Untyped methods

This PR introduces 1 partially typed method, and clears 1 partially typed method. It increases the percentage of typed methods from 61.72% to 61.79% (+0.07%).

Partially typed methods (+1-1)Introduced:
sig/datadog/di/code_tracker.rbs:16
└── def iseqs_for_path_suffix: (String suffix) -> untyped
Cleared:
sig/datadog/di/code_tracker.rbs:14
└── def iseqs_for_path_suffix: (String suffix) -> untyped

If you believe a method or an attribute is rightfully untyped or partially typed, you can add # untyped:accept on the line before the definition to remove it from the stats.

@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Apr 3, 2026

Benchmarks

Benchmark execution time: 2026-04-03 21:09:19

Comparing candidate commit a14bf0d in PR branch di-integration-test-stdlib-probes with baseline commit 609f934 in branch master.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 44 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:method instrumentation

  • 🟥 throughput [-12052.961op/s; -11217.031op/s] or [-7.913%; -7.364%]

@datadog-datadog-prod-us1-2
Copy link
Copy Markdown

datadog-datadog-prod-us1-2 bot commented Apr 3, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 98.67%
Overall Coverage: 95.37% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 02905d6 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Generated Largely based on code generated by an AI or LLM. This label is the same across all dd-trace-* repos dev/testing Involves testing processes (e.g. RSpec)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants