DI: add re-entrancy guard for method probes on stdlib#5560
DI: add re-entrancy guard for method probes on stdlib#5560
Conversation
When CodeTracker starts, use the all_iseqs C extension to populate the registry with instruction sequences for files that were loaded before tracking began. This enables line probes on third-party code and application code loaded at boot time. Only whole-file iseqs (first_lineno == 0) are backfilled — per-method iseqs require instrumenter changes to select the correct iseq for a target line and will be supported in a follow-up. Backfill does not overwrite entries from :script_compiled, which are authoritative. The C extension availability is checked via DI.respond_to?(:all_iseqs) so the code gracefully degrades when the extension is not compiled. - Added CodeTracker#backfill_registry - Called from CodeTracker#start after trace point is enabled - Added RBS signature - Added tests for backfill behavior and C extension fallback Co-Authored-By: Claude <noreply@anthropic.com>
- Added rescue block around backfill_registry so failures are best-effort (logged + telemetry) rather than propagating - Replaced all skip-based tests with mock-based tests that exercise backfill logic without requiring the compiled C extension - Added tests for: mixed iseq types, multiple files, error handling, suffix/exact lookup on backfilled entries, start ordering - 27 examples, 0 failures, 0 pending, 0 skipped Co-Authored-By: Claude <noreply@anthropic.com>
Tests the end-to-end flow: test class loaded before code tracking starts → CodeTracker#start triggers backfill via all_iseqs C extension → iseq recovered from object space → line probe installed on backfilled iseq → probe fires and captures local variables. Runs under rake spec:di_with_ext (requires compiled C extension). Three test cases: - Probe installs successfully on backfilled iseq - Probe fires when target line executes - Snapshot captures local variables from backfilled iseq Co-Authored-By: Claude <noreply@anthropic.com>
On macOS CI the C extension is compiled, so backfill_registry populates the CodeTracker registry with pre-loaded files during start. This broke existing tests that expect the registry to be empty after start or to contain exactly N explicitly-loaded files. Fix by stubbing backfill_registry in test contexts that exercise :script_compiled behavior. Backfill is tested separately in its own describe blocks. Affected contexts: - CodeTracker #start (before block) - CodeTracker shared context 'when code tracker is running' - CodeTracker #iseqs_for_path_suffix (around block) - Instrumenter shared context 'with code tracking' Co-Authored-By: Claude <noreply@anthropic.com>
…kfill The backfill filter used first_lineno == 0 to identify whole-file iseqs, but most whole-file iseqs from all_iseqs have first_lineno == 1. The new DI.iseq_type method reads the iseq type directly from the Ruby VM struct and returns a symbol (:top, :method, :block, :class, etc.). The backfill now filters by type == :top || type == :main, which correctly identifies whole-file iseqs regardless of first_lineno. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rb_iseq_type is an internal Ruby function that only exists in Ruby 3.1+. On Ruby 2.7 and 3.0, referencing it causes an undefined symbol error at load time, crashing the entire C extension (including all_iseqs and exception_message which work fine on those versions). Use have_func in extconf.rb to detect rb_iseq_type at compile time, and wrap the iseq_type function + registration in #ifdef HAVE_RB_ISEQ_TYPE. The Ruby code in code_tracker.rb already handles the missing method via DI.respond_to?(:iseq_type) with a first_lineno fallback. Co-Authored-By: Claude <noreply@anthropic.com>
… require - Add doc comments for rb_iseqw_new and rb_iseqw_to_iseq prototypes in di.c (internal Ruby functions used without documentation) - Add error handling test coverage for backfill_registry: verify logger.debug is called with the error message and telemetry.report is called when DI.current_component is available - Add test coverage for the first_lineno == 0 fallback path when iseq_type is unavailable (Ruby versions without rb_iseq_type) - Add missing require "datadog/di/spec_helper" to iseq_type_spec.rb for consistency with other ext specs - Fix skip message: iseq_type availability depends on rb_iseq_type in the Ruby runtime, not on the DI C extension Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- di.c: Document that rb_iseq_type was added in Ruby 3.1, explain the HAVE_RB_ISEQ_TYPE compile-time guard, and note the fallback path - code_tracker.rb: Replace "first_lineno == 0" YARD doc with full description of both strategies (iseq_type on 3.1+, first_lineno heuristic on older Rubies) and their tradeoffs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The YARD doc claimed the first_lineno == 0 fallback "can match top-level eval iseqs" but this is wrong. InstructionSequence.compile passes first_lineno = 1 (not 0), and require/load passes INT2FIX(0) in Ruby's rb_iseq_new_top/rb_iseq_new_main. Both strategies produce the same result in practice. Verified by reading Ruby 3.0 source (iseq.c lines 813-822): rb_iseq_new_with_opt(ast, name, path, realpath, INT2FIX(0), ...) → ISEQ_TYPE_TOP with first_lineno = 0 And compile path (iseq.c line 1064): rb_iseq_new_with_opt(&ast->body, label, file, realpath, line, ...) → line defaults to INT2FIX(1) for compile/eval Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verify idempotency: calling backfill_registry a second time with the same iseqs doesn't duplicate entries (registry.key? guard). Also verify that a second call with new iseqs adds them without overwriting entries from the first call. Co-Authored-By: Claude <noreply@anthropic.com>
The guard was purely defensive — the C extension is always compiled when DI is active (enforced by environment_supported? in component.rb). The rescue block at the bottom of backfill_registry already catches any exception if file_iseqs fails, making the guard redundant. Co-Authored-By: Claude <noreply@anthropic.com>
The method is called for side effects only. Without the explicit nil, the happy path leaked the synchronize return value and the rescue path leaked the telemetry report return value. Co-Authored-By: Claude <noreply@anthropic.com>
On older Rubies, accessing an uninitialized instance variable via &. produces a warning: "instance variable @current_components not initialized". This triggers loading_spec failures because datadog/di/preload produces unexpected output. The variable is accessed by DI.current_component (called from backfill_registry's error boundary) before any component is added. Initializing to nil at module level suppresses the warning while preserving the existing lazy-init behavior in add_current_component. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
RSpec's verify_partial_doubles rejects allow(DI).to receive(:iseq_type) when the method doesn't exist on the module. On Ruby < 3.1, rb_iseq_type is not available so DI.iseq_type is never defined. Fix: conditionally stub iseq_type only when it exists. On older Rubies, let respond_to?(:iseq_type) return false naturally and exercise the first_lineno == 0 fallback path — which is what production does. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The pre-loaded test class's iseq can be garbage collected before backfill walks the object space, causing DITargetNotInRegistry. In production, application code is referenced by live constants/methods and survives GC. In the test, the iseq is more ephemeral. Disable GC around activate_tracking! (which calls backfill_registry) to ensure the iseq is still in the object space when all_iseqs runs. Re-enable immediately after. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two root causes: 1. code_tracker_spec.rb: iseq_type was stubbed with and_call_original, but the C function expects a real RubyVM::InstructionSequence, not a test double. Stub returns :top for first_lineno==0, :method otherwise. 2. backfill_integration_spec.rb: The top-level file iseq (first_lineno=0, type=:top) is not referenced by any constant or method after loading. GC could collect it between require_relative (file load time) and the before block's backfill_registry call. Move GC.disable to file level, immediately before require_relative, so the iseq survives until backfill walks the object space. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
The ivar is initialized to nil to avoid Ruby 2.6/2.7 warnings. RBS type needs to reflect this. Silence false positive on << after ||= (Steep doesn't track that ||= guarantees non-nil). Co-Authored-By: Claude <noreply@anthropic.com>
Root cause: the top-level (:top) iseq for the test class file has no references after loading completes — only class/method child iseqs survive via BackfillIntegrationTestClass. The previous approach disabled GC at file load time and re-enabled it in the before block after backfill. This protected the first test, but after deactivate_tracking! cleared the registry (the only reference to the iseq), GC could collect it before the next test's backfill_registry walked object space. Fix: capture the top-level iseq in a constant (BACKFILL_TEST_TOP_ISEQ) immediately after loading, before GC can collect it. The constant keeps the iseq alive for the lifetime of the process, so backfill_registry can find it in every test regardless of GC activity. Verified: 0 failures across 8 consecutive full DI suite runs (714 examples each), vs ~20% failure rate before the fix. Co-Authored-By: Claude <noreply@anthropic.com>
Add "Iseq Lifecycle and GC" section to DynamicInstrumentationDevelopment.md covering: iseq types created on file load, which survive GC and why, implications for backfill_registry, and the correct test pattern for keeping top-level iseqs alive across tests. Add cross-reference from code_tracker.rb backfill_registry docstring. Co-Authored-By: Claude <noreply@anthropic.com>
When a file's whole-file (:top) iseq has been garbage collected, per-method iseqs from all_iseqs can still be used to target line probes. This covers 86% of files that were previously untargetable. Changes: - backfill_registry stores per-method iseqs in per_method_registry (grouped by path) instead of discarding them - New iseq_for_line(suffix, line) method tries whole-file iseq first, then searches per-method iseqs for one whose trace_points include the target line - Instrumenter uses iseq_for_line when available, falls back to iseqs_for_path_suffix for compatibility Verified: 37 code_tracker tests pass, lint clean, types clean. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Loads a test class, GCs the top iseq, then verifies that the backfill finds the surviving method iseq and a line probe can be installed, fired, and captures local variables through it. Precondition checks skip the test if GC didn't collect the top iseq or if the C extension is unavailable. Verified: 3 integration tests pass (install, fire, capture locals). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The throwable now includes a stacktrace array (from the C extension commit). Also update error message assertion for the new raise_if_probe_in_loaded_features format. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The raise_if_probe_in_loaded_features now reports whether per-method iseqs exist or not, instead of the generic "not in code tracker registry" message. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Distinguish between "has per-method iseqs but none cover this line" and "has no surviving iseqs at all". Include the target line number in the error. Helps users understand why a line probe failed and whether the file is partially targetable. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The method gained line_no and code_tracker parameters in the per-method iseq support commit, but the RBS declaration was not updated. Co-Authored-By: Claude <noreply@anthropic.com>
On Ruby < 3.1, DI.iseq_type is not defined (the C extension gates it with have_func for rb_iseq_type). RSpec's partial double verification rejects stubs for undefined methods, causing 10 failures in the iseq_for_line and per-method iseq tests. Fix: guard iseq_type stubs with `if Datadog::DI.respond_to?(:iseq_type)`, matching the existing pattern in backfill_registry tests (lines 246-255). On Ruby < 3.1, the code under test falls back to first_lineno == 0. Co-Authored-By: Claude <noreply@anthropic.com>
…essing Test DI's behavior when probes target standard library methods that DI itself calls internally (re-entrancy scenarios). Key findings: - Capture probes (rate limit 1/sec) on String#length, Hash#each, Array#map, Object#instance_variables: rate limiter prevents infinite recursion — nested invocations from DI processing are rate-limited and call the original method via super. - Non-capture probes (rate limit 5000/sec) on String#length: SystemStackError — the high rate limit allows recursive invocations via SecureRandom.uuid -> gen_random_urandom -> String#length -> probe -> build_snapshot -> SecureRandom.uuid -> ... The stack overflow occurs inside rate_limiter.allow? before the rate limit can stop it. - Line probe on Set#include? (set.rb): works with untargeted trace points since set.rb is loaded before code tracking starts. - Method probe on Module#prepend: fires during its own installation but does not cause infinite recursion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… vulnerability Line probes (TracePoint) are NOT vulnerable to re-entrancy because Ruby self-disables a TracePoint during its callback. Method probes (module prepending) ARE vulnerable because prepend has no such protection. New tests: - Set#include? method probe with capture: safe (rate limiter) - Set#include? method probe without capture: safe (not in snapshot path) - Set#include? line probe without capture: safe (TracePoint self-disabling) Updated file header with comprehensive findings summary. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents the desired behavior that probes on stdlib methods should only fire for customer code, not DI-internal processing (add_probe, serialization, snapshot building, flush, shutdown). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevents SystemStackError when probes target stdlib methods that DI calls internally (e.g., String#length via SecureRandom.uuid during snapshot building). The guard uses fiber-local storage (Thread.current[:datadog_di_in_probe]) and is held only during DI processing phases (pre-processing and post-processing), released during super() so recursive calls and nested probes on other methods fire normally during user code. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Typing analysisNote: Ignored files are excluded from the next sections.
|
BenchmarksBenchmark execution time: 2026-04-03 21:09:19 Comparing candidate commit a14bf0d in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 44 metrics, 1 unstable metrics.
|
|
✅ Tests 🎉 All green!❄️ No new flaky tests detected 🎯 Code Coverage (details) 🔗 Commit SHA: 02905d6 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback! |
What does this PR do?
Adds a fiber-local re-entrancy guard to method probes and integration tests exercising it.
When a probe targets a stdlib method that DI calls internally during snapshot building (e.g.,
String#lengthviaSecureRandom.uuid), the probe would re-enter DI processing recursively. The guard prevents this by skipping DI processing for calls that occur while DI is already processing a probe.The guard is a split guard: held during DI pre-processing (serialization) and post-processing (notification), but released during
super()so that recursive calls and nested probes fire normally during user code execution.Motivation:
Without this, a non-capture probe on
String#lengthcausesSystemStackErrorvia:probe fires → build_snapshot → SecureRandom.uuid → gen_random_urandom → String#length → probe fires → ...Change log entry
Yes. Dynamic Instrumentation: fix SystemStackError when probes target standard library methods used internally by probe processing.
Additional Notes:
The guard uses
Thread.current[:datadog_di_in_probe](fiber-local, not thread-local) so each fiber tracks independently — correct for fiber-based web servers (Falcon, async).Line probes are not affected because Ruby's TracePoint is self-disabling during its callback.
How to test the change?
Automated:
135 examples, 0 failures.
Gobo (manual verification via Live Debugger UI):
Stringclass,lengthmethod — this is the C-implemented stdlib method that triggered the SystemStackErrorString#lengthis called pervasively by Ruby codeSystemStackError: stack level too deep