Allow to custom timeout for patroni cluster api

[wsxyeah]

When the patroni node down, patroni cluster api may not responding tcp RST, while leaving the http request hanging.

Setting a custom timeout parameter would be ok for this case.

pg_doorman/src/bin/patroni_proxy/patroni.rs

Line 8 in a8e3064

const HTTP_TIMEOUT: Duration = Duration::from_secs(5);

[vadv]

Thank you @wsxyeah for raising this issue! You're absolutely right to point out this problem.

To address the hanging HTTP requests when a Patroni node is down, I agree that setting connect_timeout when creating the HTTP client would be a good solution. While I'm not deeply familiar with all the nuances of reqwest, setting an explicit connection timeout should help ensure that we don't hang indefinitely waiting for a TCP connection to establish.

As you can see in the current code, we already add hosts to the blacklist when they fail to respond. This prevents us from repeatedly hammering unresponsive nodes and ensures we move on to other cluster members quickly.

Additional considerations:

I agree with your point about the underlying issue. The Patroni cluster API returns consistent cluster state from the DCS (Distributed Configuration Store), but Patroni itself may experience communication problems with the DCS. In such cases:

1. 5 seconds is a reasonable timeout for most scenarios, as it allows sufficient time for Patroni to respond under normal conditions while still being responsive enough to detect genuinely unavailable nodes.

2. The distinction between node failures and DCS latency is important: A slow DCS (where Patroni is alive but experiencing communication delays) is a different problem than a completely down node, and the timeout should handle both cases gracefully.

Making timeouts configurable:

I think moving these timeout parameters to the configuration file is a great idea.