Confusion over setting timeout and max_retries

[scottyhq]

The PlanetaryComputer API has been especially finicky for me today, giving the opportunity to try upping the default timeout and max_retries on a search. But these are not having a discernible effect.

It looks like timeout can be set either as a keyword to client.open() or as an input to StacApiIO(), could these be in conflict?

pystac-client/pystac_client/client.py

Lines 102 to 103 in 7cb5fd2

	stac_io: StacApiIO | None = None,
	timeout: Timeout | None = None,

pystac-client/pystac_client/stac_api_io.py

Lines 48 to 49 in 7cb5fd2

	timeout: Timeout | None = None,
	max_retries: int | Retry | None = 5,

I was also expecting these two settings to work together (e.g. if we exceed a timeout, a retry will be triggered up to max_retries). But maybe retries are only attempted for specific return codes? Or maybe they only apply to client.search() and not subsequent commands like retrieving the item_collection?

Example:

import pystac_client 
from pystac_client.stac_api_io import StacApiIO
url = 'https://planetarycomputer.microsoft.com/api/stac/v1'
client = pystac_client.Client.open(
        url=url, stac_io=StacApiIO(max_retries=5), timeout=60,
)

%%time 

# This code occasionally fails with "APIError: The request exceeded the maximum allowed time, please try again. If the issue persists, please contact planetarycomputer@microsoft.com."

# fast
results = client.search(
    collections=["cop-dem-glo-30"],
    bbox=[-107.989298, 38.143321, -106.13571, 39.389549],
)  

# sporadic timeout at 30s
ic = results.item_collection() 

I came across this related over in pystac, but I'm still confused
stac-utils/pystac#1353

[jsignell]

It looks like timeout can be set either as a keyword to client.open() or as an input to StacApiIO(), could these be in conflict?

If you define a stac_io object and set a timeout as a kwarg in Client.open, the stac_io object will be updated to have the new timeout. So the one in Client.open will win.

I was also expecting these two settings to work together (e.g. if we exceed a timeout, a retry will be triggered up to max_retries). But maybe retries are only attempted for specific return codes? Or maybe they only apply to client.search() and not subsequent commands like retrieving the item_collection?

It does look like the timeout and the retires are set up to work on every request. The max_retries goes into the session that is then used within self.request and timeout is also used by self.request.

It's totally possible that something odd is going on in pystac-client, but my first impulse is that the planetary computer endpoint might have been unhealthy when you were testing this, or you maybe you were getting rate limited.

Does signing the request improve things at all? Something like:

import pystac_client 
import planetary_computer

 pystac_client.Client.open(
    "https://planetarycomputer.microsoft.com/api/stac/v1",
    modifier=planetary_computer.sign_inplace,
)