✨ Allow opt-in for client-go REST client metrics#3510
Conversation
k8s-ci-robot
added
the
cncf-cla: yes
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: richabanker The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
size/L
|
@richabanker: The following test failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
| prometheus.HistogramOpts{ | ||
| Name: "rest_client_request_duration_seconds", | ||
| Help: "Request latency in seconds. Broken down by verb and host.", | ||
| Buckets: []float64{0.005, 0.025, 0.1, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0, 15.0, 30.0, 60.0}, |
There was a problem hiding this comment.
Can we support native histograms from the get-go?
There was a problem hiding this comment.
(see the native histogram config for controller_runtime_reconcile_time_seconds)
| // restMetricsEnabled gates the opt-in adapters below, they discard observations | ||
| // until RegisterDefaultRESTClientMetrics flips it. requestResult stays on | ||
| // unconditionally to preserve existing behavior. | ||
| var restMetricsEnabled atomic.Bool |
There was a problem hiding this comment.
Would it help readability if we used an atomic pointer that switches the entire adapterset rather than having a check in each method? Is there a no-op from client-go we can reuse?
| "testing" | ||
| "time" | ||
|
|
||
| "github.com/stretchr/testify/require" |
There was a problem hiding this comment.
I would prefer if we can avoid introducing testify just for two checks. Let's align to our existing tests please
sbueringer
changed the title
| var registerDefaultsOnce sync.Once | ||
|
|
||
| // RegisterDefaultRESTClientMetrics enables the client metrics. | ||
| func RegisterDefaultRESTClientMetrics() { |
There was a problem hiding this comment.
Could we make this more flexible so it's possible to specify exactly which metrics to enable?
In Cluster API for example we probably cannot enable all of these metrics because at scale we're communicating with 1k+ different apiservers. So it would be nice to be able to enable only specific metrics.
(not sure if there's prior art somewhere for on API that allows to enable a set of metrics)
(cc @alvaroaleman)
4 participants
Introduce opt-in mechanism for client-go REST client metrics
These metrics were previously not supported by default due to high cardinality risks (ref: #2298)
This PR introduces an opt-in mechanism allowing downstream users to selectively enable these metrics if needed, while still preventing
sync.Onceconflicts by registering the adapters early.