Use exponential backoff when hitting too many requests responses #354
Description
Which version of integration_openai are you using?
latest
Which version of Nextcloud are you using?
latest
Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.
Describe the Bug
When the OpenAI API responds with Too many requests due to an imposed rate limit that is hit, the app currently just fails the task.
Expected Behavior
integration_openai should retry the request employing exponential backoff with jitter. ie. After hitting a too many requests response, we retry 2 more times, first after a random amount of seconds between 5 and 20s (ymmv), second time after a random amount of seconds between 60 and 120s (ymmv).
To Reproduce
Use IONOS API and send more than 5 tasks per second.